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PREFACE 


HOW  TO  USE  THE  PROJECT  MANAGER’S  GUIDE 

The  Guide  is  organized  into  two  parts.  Part  A  (chapters  I  to  VIII)  is  a  chrono¬ 
logical  guide  to  project  management;  part  B  (chapters  IX  to  XXII)  is  a  guide  to  the 
key  disciplines  encountered  in  implementing  part  A  Each  chronological  step  depends 
on  the  previous  steps  and  largely  determines  future  directions,  so  you,  the  user,  are 
encouraged  to  become  familiar  with  part  A  in  its  entirely.  Part  B  is  intended  to  pro¬ 
vide  useful  advice  even  to  the  specialist  in  the  particular  discipline;  in  usingpart  A,  it 
will  become  necessary  to  refer  to  sections  of  part  B.  Although  the  procedures  and 
guidance  provided  comform  to  “the  system,”  efforts  have  been  made  to  describe  the 
most  effective  and  efficient  means  to  each  end;  therefore,  some  of  the  concepts  pre¬ 
sented  are  nontraditional  and  may  be  new  to  the  user. 

Should  questions  or  comments  arise  in  using  this  manual,  please  contact  John 
Townsend  by  mail  or  telephone. 

Commander,  Naval  Ocean  Systems  Center 

Attention  JH  Townsend 

271  Catalina  Boulevard 

San  Diego,  CA  92152 

Autovon:  553-1382  Commercial:  (619)  553-1382 

This  guide  is  the  product  of  a  continuing  effort;  your  comments  are  welcomed  toward 
improving  its  content  and  utility. 

ADMINISTRATIVE  BACKGROUND 

This  guide  was  produced  by  the  Engineering  and  Computer  Sciences  Depart¬ 
ment,  Naval  Ocean  Systems  Center,  under  the  auspices  of  the  Center’s  Low  Cost 
Electronics/TELCAM  project.  Low  Cost  Electronics  is  an  acquisition  R&D  project 
which  is  studying  and  making  recommendations  on  ways  to  obtain  better  electronics 
equipments  while  lowering  their  total  costs.  TELCAM  (Telecommunications  Equip¬ 
ment  Low  Cost  Acquisition  Method)  addresses  the  use  of  existing  commercial  and 
military  equipments  in  new  military  applications.  Both  portions  of  the  project  were 
responses  by  the  Center  to  the  Electronics  “X”  study'  and  others.  The  project  empha¬ 
sis  has  been  to  implement  practical  methods  of  acquiring  military  electronics. 

Numerous  dedicated  people  from  commands  of  the  Navy  Department  and  also 
the  Army  and  Air  Force,  from  offices  of  the  Department  of  Defense,  from  the  defense 
industry,  and  from  commercial  industry  have  made  substantial  contributions  to  the 
effort  since  its  inception  in  1973.  Special  thanks  go  to  the  faculty  and  students  of  the 
Naval  Post-graduate  School  for  their  outstanding  support. 


'“Electronics  X;  A  Study  of  Military  Electronics  with  Particular  Reference  to  Cost 
and  Reliability,”  Institute  for  Defense  Analyses  Report  R-195,  January  1974 
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PART  A:  CHRONOLOGICAL  GUIDE 


I.  INTRODUCTION 

THE  PATTERN  OF  SUCCESS 


The  role  of  the  project  manager  is  to  acquire  equipments  which  will  perform 
the  required  functions  at  an  affordable  price  and  by  the  time  they  are  needed.  In 
these  days  of  constrained  budgets,  “affordable*  may  be  defined  as  the  least  total  cost 
to  the  government  only  with  the  proviso  that  the  functions  to  be  performed  are  worth 
that  expenditure.  Literally  hundreds  of  studies  have  looked  at  defense  acquisitions 
over  the  past  50  years.  Reading  the  conclusions  and  recommendations  from  a  1949 
report  is  like  reading  the  results  of  a  1974  report.  Project  after  project  fails  to  achieve 
these  goals.  Each  report  is  clear;  projects  fail  to  meet  performance  objectives,  overrun 
costs  by  150%,  and  slip  schedules  25-50%  or  more.  In  the  vast  majority  of  cases,  the 
project  goals  were  not  achieved  for  the  following  reasons: 

•  misspeciflcation  (usually  gross  over-specification  creating  artificial 
technical  problems) 

•  failure  to  manage  risks 

•  obscuring  of  the  project  goals  through  extraneous  paperwork  requirements 

•  failure  to  adequately  define  what  is  required 

•  underestimating  the  required  project  resources  (sometimes  intentionally, 
in  order  to  “buy  in”  and  get  a  project  started) 

These  reasons  are,  however,  only  the  symptoms  of  underlying  problems  in  the 
acquisition  community.  The  TELCAM  project  looked  at  successes  and  failures  in  in¬ 
dustry  as  well  as  government;  the  successful  project  has  the  same  traits  whether  in 
industry  or  in  government.  An  acquisition  project  also  shares  many  traits  with  a 
small  business,  so  TELCAM  also  solicited  information  from  the  Small  Business  Ad¬ 
ministration.  Again,  success  is  a  pattern,  whereas  failure  is  a  deviation  from  that  pat¬ 
tern.  The  mqjor  difference  between  failures  in  industry  and  failures  in  government  is 
that  a  failing  project  in  industry  is  usually  rapidly  terminated;  the  government  failure 
usually  plods  on  to  an  elegant  wreck. 

What  is  the  elusive  pattern  of  success?  The  projects  cited  as  successes  will 
have  two  main  features: 

•  A  strong,  knowledgeable  project  manager  who  acts  as  the  ultimate  author¬ 
ity  for  all  project  matters  —  tasking,  budgeting,  technical  decisions. 

•  A  small,  dedicated  team  executing  project  tasks. 
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The  key  words  above  are:  strong1,  knowledgeable,  ultimate  authority,  small, 
dedicated,  and  team.  Excellent  studies  of  the  nuclear  power  program1,  the  Polaris  sys¬ 
tem,*  and  NTDS>  are  available  which  show  these  forces  at  work.  “■Strong'  appears  to 
be  a  peculiar  necessity  in  the  government  projects,  as  each  success  seems  to  attain 
that  status  in  spite  of ‘the  system.”  An  ultimate  goal  of  acquisition  R&D  must  be  to 
change  “the  system’  to  allow  average  individuals  to  be  successful  project  managers. 
Until  that  goal  is  reached,  there  is  still  enough  of  a  task  to  create  knowledgeable 
managers.  Some  spectacular  failures  have  been  managed  by  strong,  unknowledgeable 
individuals.  A  project  needs  a  strong  champion  in  order  to  ‘steal’  sufficient  authority 
to  become  a  purposeful  autonomous  entity,  but  authority  unwisely  wielded  is  disas¬ 
ter.  The  government  project  manager  is  not  held  accountable  for  his  actions;  account¬ 
ability  is  the  “quality  assurance”  incentive  used  to  check  authority  in  industry. 

In  addition,  most  successful  project  teams  were  found  to  be  goal-oriented  and 
opportunistic.  Individual  team  members  were  confident  of  success,  in  spite  of  great 
technical  difficulties,  and  really  wanted  the  project  to  succeed.  In  contrast,  team 
members  of  unsuccessful  and  stagnant  projects  focused  on  problems  and  mired  down 
into  details.  The  attitudes  of  the  team  members  reflected  the  attitudes  of  the  project 
leadership. 

The  first  procurement  of  muskets  by  the  Army  in  1798  seemed  straightfor¬ 
ward.  Eli  Whitney  promised  to  produce  and  deliver  muskets  built  by  assembly  line 
techniques,  and  using  interchangeable  parts,  within  8  months;  actual  delivery  was 
made  10  years  late.  Our  military  procurement  problems  actually  predate  the  nation 
itself;  one  verse  of  “Yankee  Doodle”  went 

“And  there  we  saw  a  thousand  men 

As  rich  as  Squire  David, 

And  what  they  wasted  every  day 

I  wish  it  could  be  saved” 

referring  to  one  of  General  Washington’s  encampments. 

On  27  March  1794,  Congress  appropriated  $768,000  for  the  construction  and 
manning  of  six  frigates.  Each  frigate  was  to  cost  $100,000.  When  the  UNITED 
STATES,  CONSTITUTION,  and  CONSTELLATION  were  finally  launched  in  1797, 
each  cost  close  to  $300,000.  The  national  budget  in  1797  was  about  $5.7  million. 

The  innumerable  instances  of  procurement  problems  which  have  occurred 
repeatedly  over  the  years  have  only  worsened  with  time.  The  evolving  acquisition  sys¬ 
tem  operated  in  ignorance  in  the  1790s,  and  this  basic  ignorance  exists  today 
throughout  the  acquisition  community.  Less  than  30%  of  the  project  managers  inter¬ 
viewed  by  TELCAM  knew  the  operational  objective  of  their  project;  only  5%  could 
relate  technical  features  of  the  project  to  operational  considerations.  Only  2%  of  the 
project  managers  were  aware  of  any  actions  being  taken  on  their  project  to  reduce 


'Nuclear  Navy  (1946-1962),  RG  Hewlett  and  F.  Duncan,  Chicago,  1974 

*The  Polaris  System  Development,  Sapolsky,  Harvard,  1972 

*The  NTDS  Development,  RW  Graf,  United  Research,  Inc.,  29  Jan  1964 


risks  of  any  kind;  only  half  knew  what  progress  was  being  made  or  what  difficulties 
were  being  encountered  on  mqjor  tasks!  Under  these  circumstances,  it  is  a  wonder 
even  greater  failures  do  not  occur  than  those  already  found.  One  of  the  mqjor  factors 
aggravating  this  situation  is  the  lack  of  project  manager  accountability  for  the  end 
item  in  the  field.  A  project  manager  may  only  influence  4  years  of  project  life, 
whereas  the  end  item  may  be  in  use  for  20-50  years.  The  project  manager  determina¬ 
tions  affect  all  but  a  few  percent  of  the  total  life  costs  of  a  project. 

Figure  1-1  shows  the  percentage  of  total  ownership  costs  committed  during 
conceptual  planning,  design,  development,  acquisition,  and  operations  for  past  major 
programs.  In  the  past,  decisions  made  during  the  concept  and  planning  phases  com¬ 
mitted  70%  of  the  total  life-cycle  cost  funds  of  a  program,  and  design,  development, 
acquisition,  and  operations  accounted  for  only  30%  of  that  total  cost.  Notice  that  only 
about  1%  of  the  life-cycle  costs  are  expended  during  this  conceptual  phase.  The  effects 
of  applying  system  engineering  principles,  including  life-cycle  cost  analysis  through 
the  planning  and  RDT&E  phases  of  a  program  and  the  “design  to  cost”  concept  are 
expected  to  change  this  distribution  considerably  by  affecting  a  larger  portion  of  that 
early  70%  commitment. 


YEARS  AND  PHASES  OF  SYSTEM  EVOLUTION 

Figure  1-1.  Systems  funds  committed  by  Initial  planning 
decisions. 

Notice  that  90%  (or  more)  of  the  total  project  cost  is  fixed  after  only  10%  of 
the  funds  have  been  expended.  Unless  the  project  manager  assumes  responsibility  for 
the  total,  it  is  impossible  to  attain  the  lowest  possible  project  cost.  Considering  that 
support  costs  mav  be  altered  hv  ns  much  as  two  orders  of  magnitude  bv  decisions  in 
the  conceptuel  phase,  it  can  be  seen  that  a  naive  approach  to  project  management  can 
have  a  devastating  effect  on  operating  budgets;  likewise,  sound  management  can  lead 
to  a  highly  efficient  use  of  funds. 
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The  acquisition  manager  should  try  to  obtain  equipments  which  are  fully 
capable  of  doing  the  required  job  and  which  have  the  following  characteristics: 

Reliable 

Maintainable 

Supportable 

Procurable 

Producible 

The  remainder  of  the  Guide  discusses  the  many  facets  of  these  factors. 

THE  PROJECT  MANAGER 


A  manager  and  a  project  are  established  for  a  single  purpose.  It  makes  no  dif¬ 
ference  that  the  project  is  designated  a  mqjor  program  and  its  manager  a  program 
manager  or  that  the  project  is  called  a  tasking  with  a  project  engineer  in  charge.  Pro¬ 
gram  manager,  acquisition  manager,  project  manager  are,  for  the  purposes  of  this 
guide,  synonymous,  being  different  in  scope  but  like  in  character.  Project  manage¬ 
ment  is  the  planning,  executing,  directing,  and  controlling  of  a  relatively  short-term 
project  or  systems-oriented  organization  established  for  the  completion  of  specific 
goals.  In  this  guide,  those  specific  goals  will  be  an  acquisition  and  implementatiou  'f 
militaiy  equipments  and  the  subgoals  associated  with  each  phase  of  the  cradle-to- 
grave  of  those  equipments;  however,  the  principles  presented  are  basic  and  universal 
and  adaptable  to  many  other  project  circumstances. 

The  project  manager  ideally  will  plan,  organize,  monitor,  and  direct  the  pro¬ 
ject  to  its  goals  as  effectively  as  possible.  Efficiency  is  a  secondary  consideration, 
since  maximum  efficiency  often  compromises  effectiveness.  It  is  generally  agreed  that, 
in  the  competitive  atmosphere  of  military  affairs,  ineffectiveness  is  catastrophic. 
Organizations  which  manage  for  efficiency  are  called  functional  organizations.  In 
executing  his  tasks,  the  project  manager  will  draw  on  expert  assistance  from  many 
functional  areas  and  will  establish  lines  of  organization  control  which  will  allow  him 
to  manage  efficiently.  In  general,  he  has  two  cardinal  rules  to  follow: 

«  Do  not  do  it  yourself  —  accomplish  through  the  project  organization. 

•  Organize  your  resources  to  fit  the  project  —  be  prompt  and  precise  in 
defining  the  organization. 

The  project  organization  exists  within  an  organization  (which  will  normally  be 
functionally  organized).  In  order  to  reach  its  goals,  it  must  live  within  the  chain  of 
command  of  its  parent  organization,  and  it  must  establish  a  chain  of  command  within 
itself.  A  chain  of  command  is  an  organization  of  three  elements:  responsibility, 
authority,  and  accountability.  Usually  a  project’s  charter  will  define  the  project  goals 
without  mention  of  these  throe  elements.  Organization  instructions  may  define  pro¬ 
ject  manager  responsibilities  in  a  general  way  with  only  implications  of  assigning 
authority  and  no  actual  accountability.  In  practice,  the  project  manager  should 
assume  that  he  is  fully  responsible  for  meeting  his  project  goals  and  he  should  assume 
all  the  authority  that  he  is  allowed  by  law  and  by  his  supervision  to  meet  his  respon¬ 
sibilities.  Within  the  project  organization,  he  will  clearly  reassign  responsibilities, 
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delegate  appropriate  authority,  and  hold  accountable' each  responsible  individual.  The 
key  to  his  success  will  often  be  his  authority  and  his  ability  to  exercise  and  delegate 
it.  Outside  the  project,  the  manager  should  elicit  the  cooperation  of  those  who  have 
authority  above  him,  to  ensure  that  he  is  backed  up,  by  keeping  his  chain  of  com¬ 
mand  informed  truthfully,  concisely,  and  specifically.  Authority  is  the  power  to  make 
decisions.  It  is  important  to  remember  that  small  decisions  must  be  made.  A  “no  deci¬ 
sion*  is  worse  than  a  “wrong  decision’’  because  with  the  wrong  decision  the  manager 
knows  what  he  did  and  can  correct  it;  with  no  decision,  the  situation  will  inevitably 
grow  worse,  perhaps  without  any  indication  of  the  appropriate  corrective  action. 
Admirid  Nimitz  was  reputed  to  have  said,  ‘the  time  for  taking  all  means  for  a  ship’s 
safety  is  while  you  are  still  able  to  do  so.”  Decisions  are  required  to  solve  problems; 
solutions  usually  result  from  perspiration  —  not  inspiration.  When  a  manager  has  a 
problem,  he  has  basically  three  methods  available  to  solve  it;  the  important  thing  is 
that  the  decision  be  made. 


PROBLEM  SOLVING 


Classical  Method  Scientific  Method  Shotgun  Method 


What  is  the  problem? 

1.  Define  the  problem 

1.  Make  decision 

What  are  the  alternatives? 

2.  State  objectives 

2.  Analyze  result 

What  is  the  best  alterna¬ 
tive? 

3.  Formulate  a  hypothesis 

3.  Correct 

4.  Collect  data 

4.  Implement 

5.  Classify,  analyze,  and 
interpret  data  against 
thehypo  thesis 

5.  (repeat  2-4  as 
required) 

6.  Draw  conclusions,  generalize, 
restate,  or  develop  new 
hypotheses. 

The  solution  should  be  kept  in  perspective  by  asking,  “Is  it  adequate?”  and  “Is  it  too 
elaborate?”  These  questions  are  asked  in  the  context  of  the  ultimate  project  goals. 

In  order  to  make  decisions,  the  manager  must  be  informed.  Tho  manager  uses 
the  project  organization  and  procedures  to  keep  informed  of  project  activities,  usually 
utilizing  some  form  of  convenient  management  information  system.  Again,  the  solu¬ 
tion  is  tailored  to  the  needs.  On  small  projects,  the  project  manager  will  keep  directly 
informed  about  all  the  specifics  of  his  projoct.  On  large  projects,  the  project  manager 
will  rely  mainly  on  reports  and  plans  and  will  focus  on  exceptions  to  the  overall  plan. 

THE  SYSTEMS  ENGINEER 


Systems  engineering  is  both  an  engineering  management  discipline  and  a  tech¬ 
nical  engineering  process.  It  is  a  branch  of  engineering  devoted  to  the  design, 


I-S 


development,  and  application  of  complete  systems,  taking  into  consideration  all  ele¬ 
ments  in  a  system  or  process  and  their  integration.  A  system  is  an  integrated  assem¬ 
blage  of  elements  operating  together  to  accomplish  a  prescribed  end  purpose. 

As  an  engineering  manager,  the  systems  engineer  is  responsible  for  planning, 
organizing,  and  tracking  the  system  design  elements;  technical  performance,  internal 
and  external  interfaces,  production  and  support  cost  elements,  documentation  (and 
data  management),  configuration  management,  and  test  issues. 

As  a  deputy  (or  assistant)  project  manager,  the  systems  engineer  is  responsible 
for  risk  management  and  for. structuring  the  technical  effort  to  match  the  project 
resources  and  the  acquisition  strategy. 

As  a  technical  task  manager,  the  systems  engineer  is  responsible  for  the  sys¬ 
tem  design.  System  design  decisions  account  for  70%  of  life-cycle  costs;  total  life-cycle 
costs  may  be  affected  by  as  much  as  two  orders  of  magnitude. 

System  design  translates  operational  requirements  into  technical  specifica¬ 
tions.  The  specifications  describe  a  product  design  which  is  compatible  with  the  user 
requirements,  the  usage  environments,  the  project  resources,  the  production  require¬ 
ments,  and  the  support  environment.  System  engineering  provides  the  DISCIPLINE 
to  assure  that  the  product  is  consistent  with  the  requirements  and  with  the  other  sys¬ 
tem  design  elements. 

The  system  design  parameters  include: 

•  schedule 

•  risks 

•  supportability 

•  maintainability 

•  accessibility 

•  quality 

•  security 

•  storage  and  handling 

•  transportability 

•  project  cost/life-cycle  costs 

•  producibility 

«  reliability/dependability 

•  availability 

•  tools/ test  equipment 

•  safety 
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•  testability 

•  Hiring-  *n/t  parking 

•  EMC/EMI/EMP/EME/HERO/TEMPEST 

•  usage,  storage,  and  transportation  environments 

•  project  engineering  environment  (CIE  tools) 

•  human  factors  for  operation  and  maintenance 

•  platform  interface  and  installation  requirements 

•  available  technologies  (for  build,  buy,  modify  decisions) 

•  future  modification  requirements 

•  contractual  resources  and  relationships 

ALSO  -  DOES  IT  MEET  THE  USER’S  REQUIREMENTS? 

In  order  to  obtain  up-the-line  cooperation  to  get  higher  order  decisions,  the 
project  manager  and  systems  engineer  should  know  their  own  project  and  also  related 
projects.  In  knowing  their  own  project,  the  management  can  confidently  relate  accu¬ 
rate  information  to  their  superiors.  This  confidence  and  frankness  can  play  a  role  in 
generating  trust  which  will  be  valuable  if  problems  requiring  outside  assistance  arise. 
Also,  the  knowledge  of  other  projects  will  assist  the  manager  in  recognizing  the  par¬ 
ent  organization’s  perspective  and  in  establishing  a  priority  to  obtain  the  needed  de¬ 
cision.  Avoiding  “tunnel  mindedness'  can  be  very  helpful  when  competing  for  a  share 
of  limited  organization  resources  —  especially  binding.  Avoid  “buttering  up”  reports 
to  show  only  good  news;  mqjor  problems  cannot  be  covered  up  and  will  torpedo  this 
facade.  The  project  must  satisfy  the  parent  organization’s  goals. 

KEY  CONCEPTS 


PLAN!  The  process  of  planning  the  project  is  important  in  uncovering  the 
risks  and  developing  an  acquisition  strategy.  But  don’t  forget,  a  PLAN  IS  ONLY  AS 
GOOD  AS  ITS  EXECUTION. 

Don’t  confuse  information  and  data.  You  need  good  information;  data  is  only 
the  means  of  obtaining  information. 

LEAD!  Be  a  team  leader.  Set  the  highest  professional  and  ethical  standards 
for  yourself  and  for  your  team.  Communicate. 

DELEGATE!  Choose  the  knowledgeable  experts  you  need  and  give  them  the 
responsibility  and  authority  they  need  —  then  hold  them  accountable.  Support  your 
team.  Reward  your  winners. 
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SUMMARY 


The  system  acquisition  process  is  a  bureaucratic  methodology  for  transform¬ 
ing  operational  requirements  into  a  technical  implementation.  It  is  governed  by  pro¬ 
cedures,  instructions  and  directives,  regulations,  laws,  and  the  congressional  budget 
process.  The  project  manager  must  navigate  this  bureaucratic  sea  while  maintaining 
control  of  the  project.  The  system  engineering  discipline  assists  the  project  manager 
by  providing  an  effective  system  design  most  efficiently  within  the  constraints  im¬ 
posed  by  the  acquisition  system  and  the  “real"  world. 

This  guide  is  offered  in  recognition  of  the  fact  that  most  project  managers  are 
good  technical  people  who  may  be  inexperienced  managers.  It  is  also  an  attempt  to 
offer  practical  methods  to  implement  the  recommendations  of  the  various  government 
studies  on  reducing  costs  (see  appendix  B),  many  of  which  are  not  addressed  by  direc¬ 
tives  and  instructions.  It  is  hoped  that  the  Guide  will  serve  as  a  useful  navigational 
tool  for  the  manager  as  he  or  she  weaves  the  project’s  course  through  instructions, 
budgets,  specifications,  and  the  like  to  a  successful  implementation  in  the  Fleet. 
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II.  REQUIREMENTS  DEFINITION 


Hie  most  basic  step  in  any  successful  endeavor  is  to  thoroughly  estab¬ 
lish  a  goal.  This  chapter  cites  the  elements  necessary  to  define  the  goal 
for  an  acquisition  project.  These  dements  determine  how  an  item  will 
be  used  and  supported  and  how  much  it  will  cost  through  its  life  cycle. 

All  the  steps  which  follow  must  point  toward  the  project  goal. 

A  project  which  is  not  confined  to  basic  research  has  objectives  which  must 
ultimately  be  application  oriented.  The  identified  application  is  defined  in  terms  of 
user  requirements.  Usually,  these  requirements  will  be  documented  by  an  Opera¬ 
tional  Requirement  (OR)  (OPNAV  INST  5000.42).  A  project  not  having  an  OR  will 
normally  be  part  of  or  closely  related  to  a  project  which  is  so  documented;  alternately, 
a  project  whose  purpose  is  to  replace  existing  equipment  may  assume  the  same  user 
requirements  as  the  extant  equipment.  In  any  case,  certain  user-oriented  parameters 
must  be  known  before  valid  technical  requirements  can  be  generated.  The  following 
requirements  elements  should  be  identified  before  progressing  into  a  conceptual 
effort. 


Operational  need  —  the  operational  problems  and  threats  to  be  addressed  by 
the  project  including  the  scope  and  parameters  of  the  expected  threat  environment, 
deficiencies  in  present  capabilities,  and  the  consequences  of  not  satisfying  the  need; 
i.e.,  statement  of  how  vital  the  system  will  be  and  the  required  system  availability. 

Operational  concept  —  a  statement  of  how  the  user  is  to  use  the  new  capabil¬ 
ity  to  combat  the  operational  need.  This  includes  definitive  mission  parameters,  i.e., 
mission  duration,  ship-air-shore  platforms  involved,  intervals  between  missions,  the 
mission  environment,  the  system  utilization  rate,  etc.,  and  requirements  for  the  sys¬ 
tem  to  be  compatible  with  other  systems.  The  number  of  systems  to  be  implemented 
and  the  priorities  of  implementation  are  included. 

Performance  goals  —  the  known  performance  criteria,  operating  modes,  and 
other  system  characteristics  stated  with  minimum  acceptable  limits  and  allowable 
degradation  conditions. 

Life  support  goals 

Life  expectancy 

Special  logistic  and  training  support  considerations  and  constraints 

Nominal  goals  for  mission  and  operating  reliabilities 

Level  <f  maintenance  capability  constraints  including  the  maximum 
annual  maintenance  time  at  the  organizational  level 


II-l 


Cost  objectives 


Production  design-to-co6t  goals  given  the  number  of  systems  to  be 
acquired  within  the  postulated  time  frame 

The  maximum  annual  maintenance  cost  at  the  organizational  level 

Initial  operational  capability  (IOC)  —  the  desired  Fleet  introduction  date 

Additionally,  an  OR  will  state  whether  there  are  ongoing/related  efforts  which 
should  be  coordinated  with  the  project. 

If  any  of  the  above  requirements  elements  are  deficient,  efforts  should  be  initi¬ 
ated  to  obtain  the  needed  information  prior  to,  or  as  soon  as  possible  during,  the  con¬ 
ceptual  phase. 

ORIGINATION  OF  REQUIREMENTS 


Requirements  can  be  operationally  (user)  generated  or  technology-generated. 
The  acquisition  process  assumes  that  all  requirements  are  operationally  generated, 
but  most  successful  projects  are,  in  fact,  championed  by  the  technologist.  (Over  9  out 
of  every  10  projects  were  actually  technology-generated.)  Nevertheless,  the  successful 
technologist  obtains  user  support  and  at  least  maintains  the  appearance  of  opera¬ 
tional  generation  of  the  requirement.  There  are  perhaps  1000  times  more  valid  needs 
than  there  are  budgetary  resources  to  support  formal  requirements  to  address  those 
needs,  so  mutual  support  from  OPNAV  (representing  the  user  communities)  and  the 
systems  command  (representing  the  technologies)  is  needed  to  establish  a  formal  re¬ 
quirement. 

Fleet  deficiencies  are  often  identified  by  the  type  commanders  through  the  sys¬ 
tems  commands.  This  process  sometimes  loses  the  initial  operational  flavor  and  as¬ 
sumes  a  technical  solution.  This  process  may  miss  the  opportunity  to  reserve  several 
problems  beyond  the  original  statement  of  deficiency,  resulting  in  a  requirements 
statement  which  is  too  narrow  in  scope. 

The  technologist  often  “sells”  a  technology  to  the  user  community  as  a  poten¬ 
tial  solution  to  various  needs.  This  may  result  in  an  attempt  to  misapply  the  technol¬ 
ogy  to  do  more  than  it  can  do  effectively.  Also,  the  Fleet  may  not  be  able  to  develop  a 
clear  concept  of  how  the  technology  should  be  employed  or  supported.  This  results  in 
a  requirements  statement  which  is  poorly  defined  and  probably  too  broad  in  scope. 

The  source  of  the  requirement  is  important  to  the  system  designer  because 
requirements  are  never  adequately  defined.  Operationally  generated  requirements 
probably  will  not  have  realistic  cost  or  schedule  constraints  and  may  make  technical 
assumptions  which  attempt  to  violate  laws  of  physics.  Technology-generated  require¬ 
ments  probably  will  not  have  clearly  stated  concepts  of  employment  and  concepts  of 
support.  The  behind-the-scenes  maneuvering  to  generate  a  requirements  document 
which  will  be  supported  by  the  budget  process  normally  leads  to  the  omission  of  even 
the  valid  information  which  might  have  been  available  to  the  original  person  recog¬ 
nizing  the  need.  Therefore,  the  system  designer  almost  always  must  dedicate  some 
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significant  effort  towards  clarifying  the  requirements.  For  further  information,  see 
chapter  IV. 

REQUIREMENTS  ORIENTATION 


Requirements  are  operationally  oriented  or  technology-oriented.  Operationally 
oriented  requirements  include: 

•  Requirements  to  counter  new  threats 

•  Requirements  to  support  new  tactics 

Technology-oriented  requirements  include: 

•  New  capabilities  of  technology  to 
••  create  a  new  threat  potential 
••  establish  more  effective  tactics 
••  be  put  to  use  (GGB  WHIZ  factor) 

•  Capabilities  to  replace  obsolete  equipments  to 
••  reduce  the  cost  of  ownership 

••  increase  reliability  and  maintainability 
••  improve  operability 

••  improve  survivability  (especially  in  the  face  of  new  threats) 

••  provide  additional  capabilities  (faster,  more  lethal,  etc.) 

The  orientation  of  the  requirement  directly  affects  the  statement  of  require¬ 
ments  and  the  level  of  information  which  is  initially  available  to  the  system  designer. 
The  orientation  may  also  affect  the  project  priorities  and  may  restrict  the  system  en¬ 
gineering  options. 

Regardless  of  the  initial  orientation  of  the  requirements,  the  system  engineer 
should  assess  the  system  design  from  all  orientations  in  the  process  of  clarifying  the 
requirements.  The  project  manager  should  be  aware  of  the  requirements  orientation 
and  the  requirements  origination  in  order  to  properly  assess  the  “politics”  behind  the 
requirements  statement. 
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m.  PROGRAM  PLANNING 


This  chapter  provides  some  keystones  of  demonstrated  value  in 
the  organization,  planning,  and  execution  of  a  project.  The  man¬ 
ager  is  exhorted  to  consider  goals,  resources,  constraining  fac¬ 
tors,  and  risks  as  one  coherent  entity. 

Program  planning  is  an  art  which  attempts  to  assemble  a  number  of  conflicting  ele¬ 
ments  into  a  coherent,  practical  plan  leading  to  an  acceptable  result.  These  elements  are: 

Program  objectives 
Available  resources 
Constraining  factors 
Risks 

Thus,  the  program  plan  must  achieve  the  program  objectives  with  the  available  resources 
within  the  constraints  at’an  acceptable  risk. 

The  program  objectives  will  include,  as  a  minimum,  the  satisfaction  of  the  opera¬ 
tional  requirement.  If  the  requirements  are  not  sufficiently  well  defined,  any  undertaking  to 
satisfy  them  will  waste  valuable  resources.  Often  additional  objectives  will  be  manifest,  such 
as  “delivery  by  1979,”  “within  $30,000,”  or  “maintain  compatibility  with  what  we  have  now.” 
These  additional  objectives  may  not  be  wholly  consistent  with  the  “best”  solution  to  the  pri¬ 
mary  problem;  nevertheless,  they  must  be  addressed.  Sometimes  it  is  possible  to  influence 
the  statement  of  objectives.  If  so,  try  to  make  the  primary  objective  (satisfying  the  opera¬ 
tional  requirement)  as  clearly  defined  as  possible  and  minimize  other  objectives.  When  sec¬ 
ondary  objectives  are  included,  determine  how  much  flexibility  can  be  tolerated  in  meeting 
them. 


Resources  include: 

•  manpower 

•  talent 

•  facilities 

•  time 

•  funding 

•  test  equipment 

•  information,  and  the  like 

These  resources  are  the  raw  materials  from  which  the  program  is  built.  The  primary  prob¬ 
lem  is  to  determine  what  types  and  levels  of  resources  will  be  needed.  Manpower  and  talent 
together  comprise  personnel  resources,  which  are  indispensable;  they  are  cited  separately  to 
emphasize  the  fact  that  people  and  knowledge  do  not  solve  problems  —  knowledgeable  people 
solve  them.  Time  is  a  flexible  resource  which  is  normally  not  a  problem  unless  there  is  not 
enough  (as  when  a  ship  schedule  must  be  met).  Most  of  the  other  resources  can  be  tapped 
easily  enough  as  long  as  it  is  known  that  they  are  needed.  The  most  important  resources  are 
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personnel  and  funding;  most  projects  succeed  or  fail  on  the  availability  of  them.  The  typical 
failure  has  an  excess  of  manpower,  a  paucity  of  talent,  and  a  lack  of  control  on  the  allocation 
of  funds.  Less  is  better.  Too  many  people  make  a  project  unmanageable  and  obscure  valuable 
talents  from  the  tasks  to  which  they  should  be  applied.  If  one  could  give  the  program  objec¬ 
tive  to  a  single  engineer,  possessing  the  knowledge  and  talent  needed,  a  “best”  product  would 
result.  However,  most  projects  contain  complexities  that  an  individual  cannot  cope  with 
alone,  so  a  team  of  specialists  is  usually  assembled.  Nevertheless,  the  tendency  is  to  have  too 
many  “cooks”  on  the  project.  The  Sidewinder  missile,  Mirage  fighter,  and  the  Navy’s  first 
nuclear  propulsion  plant  were  all  designed  by  teams  of  fewer  than  40  individuals.  On  the 
other  hand,  the  C-5A  was  “designed”  by  over  1200  engineers  and  the  P-111  by  over  600.  The 
personnel  resource  requirement  should  be  based  on  the  talent  necessary  to  complete  the 
tasks  with  a  minimum  amount  of  manpower.  Personnel  and  funding  are  inextricably  related. 
The  level  of  funding  required  must  be  sufficient  to  obtain  access  to  the  necessary  level  of 
other  resources;  the  complexities  of  funding  are  discussed  separately  later  in  this  chapter. 

The  most  obvious  program  constraints  are  limits  on,  or  a  lack  of,  resources,  particu¬ 
larly  funding  and  time.  Constraints  are  always  present  since  no  resource  is  infinitely  avail¬ 
able,  but  the  constraints  are  not  crippling  unless  they  limit  a  resource  at  a  level  below 
sufficiency.  If  a  resource  is  constrained  below  the  level  of  necessity,  the  program  should  be 
canceled;  however,  most  constraints  simply  limit  planning  options.  Two  more  subtle  con¬ 
straining  factors  are  policy  and  politics.  Most  policies  simply  provide  guidance  or  standard¬ 
ize  procedures  and  do  not  pose  planning  problems  as  long  as  they  are  known.  Policies  which 
pose  problems  which  cannot  be  tolerated  can  be  changed  or  waivered  as  long  as  valid  justifi¬ 
cation  exists.  Request  the  change  or  waiver  from  the  originator  of  the  policy,  if  relief  is  not 
obtained,  a  solution  to  the  problem  can  generally  be  found  with  the  policymaker’s  help.  Poli¬ 
tics  can  make  or  break  a  project.  Typically,  an  otherwise  flexible  constraint  will  become  cast 
in  concrete  through  political  manipulations,  and  adverse  decisions  will  be  made  beyond  con¬ 
trol  of  the  project.  Every  situation  is  different,  and  little  guidance  can  be  offered  here  except 
to  know  the  strength  of  friends  and  the  weaknesses  of  foes  and  to  try  to  let  project  friends  do 
battle,  allowing  project  personnel  to  remain  behind  the  scenes  and  seemingly  aloof  of  the 
wars.  In  general,  it  is  desirable  to  influence  constraining  factors  to  allow  a  maximum 
amount  of  flexibility,  this  allows  the  project  manager  to  place  the  constraints  on  tasks  rather 
than  outside  influences. 

Risks  are  always  present  in  any  program,  but  they  can  generally  be  managed  and  re¬ 
duced  to  an  acceptable  level.  Project  planning  must  provide  a  framework  to  identify,  assess, 
and  act  upon  risks.  The  resources  necessary  for  these  functions  consist  of  valid  information 
and  proper  talent.  Additionally,  the  project  must  be  sufficiently  flexible  to  accommodate 
changes  which  may  be  necessary  to  manage  risks.  Risk  management  is  a  function  mandated 
to  the  project  manager  by  DoD  Directive  5000.1  and  SECNAV  Instruction  5000.1.  Some 
techniques  and  methods  for  managing  risks  are  contained  in  chapter  XXI. 

ORGANIZATION 

The  program  plan  organizes  the  program  elements  in  a  series  of  tasks  to  achieve  the 
program  objectives,  determine  resource  requirements,  and  identify  constraining  factors  and 
risks.  When  the  tasks  are  well  defined,  it  is  easier  to  make  the  estimates  required,  particu¬ 
larly  estimates  of  funding  and  schedule.  As  in  any  estimating  process,  there  are  always 
latent  estimating  errors.  Assuming  reasonable  estimating  (not  chronically  optimistic  or  pes¬ 
simistic),  estimating  errors  can  be  drastically  reduced  for  the  collective  project  by  making 
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increasingly  more  detailed  estimates  because  actual  estimating  errors  will  tend  to  cancel  as 
they  are  aggregated  and  errors  of  omission  will  tend  to  be  limited  by  the  thoughtful  uncover¬ 
ing  of  obscure  detail  tasks. 

The  standard  method  of  organizing  this  hierarchy  of  tasks  is  the  Work  Breakdown 
Structure  (WBS).  MIL-STD-881  provides  guidance  in  constructing  WBSs.  Once  constructed, 
the  WBS  becomes  a  convenient  framework  for  organizing  task  assignments,  budgeting  re¬ 
sources,  assessing  risks,  estimating  life-cycle  costs,  and  a  host  of  other  management  tasks. 
The  complexity  of  the  WBS  will  directly  reflect  the  complexity  of  the  program.  The  WBS  for 
simple  projects  can  easily  be  managed  by  hand;  for  complex  projects,  the  aid  of  a  computer  is 
useful.  NOSC,  for  example,  has  a  WBS  computer  program  for  this  purpose.  It  is  advisable  to 
create  separate  breakdowns  using  a  common  WBS  for  the  following  program  items: 

Tasks  and  resource  requirements 

Costs 

Milestones 

Risks 

Nonrecurring  and  recurring  life-cycle  costs 
Production  costs,  fixed  anavariable 
Technical  performance  measures 

In  each  structure,  it  is  advisable  to  recognize  the  design,  fabrication,  test,  assembly,  and 
documentation  tasks  associated  with  each  level  of  complexity;  such  a  breakdown  will  im¬ 
prove  the  accuracy  of  estimates  made  at  the  level  even  though  it  may  not  be  either  practical 
or  desirable  to  collect  accounting  data  at  that  level.  Refer  to  chapter  XXII  for  guidance  on 
How  to  Use  a  Work  Breakdown  Structure. 

For  a  complex  system,  a  WBS  may  contain  as  many  as  seven,  eight,  or  nine  levels. 
While  the  manager  must  be  aware  that  the  detailed  levels  exist,  he  must  also  bo  careful  not 
to  get  mired  and  saturated  in  details.  As  a  rule  of  thumb,  no  individual  should  attempt  to 
directly  manage  more  than  four  levels;  this  figure  has  been  well  established  by  empirical 
study  across  both  government  and  industry.  A  complex  project  might  be  organized  as  illus¬ 
trated  below. 


Individual  Responsible 
Project  Manager/System  Engineer 

Deputies  (one  per  group) 

Lead  Engineers 


Equipment  Breakdown  Tovel 

1.  System 

2.  Subsystem 

3.  Sets 

4.  Groups 

5.  Units 

6.  Assemblies 

7.  Subassemblies 

8.  Modules 

9.  Parts 


Of  course,  each  project  is  different,  so  the  project  organization  must  conform  to  the  project 
peculiarities.  Organizational  boundaries,  equipment  complexities,  and  technology  implica¬ 
tions  may  dictate  the  organizational  structure.  In  any  case,  the  goal  is  to  create  a  project 
organization  in  which  each  task  is  assigned  to  the  most  capable  individual  possible  giving 
him  the  responsibility  and  authority  to  execute  that  task  and  providing  a  clear-cut  chain  of 
command,  integrating  him  into  the  organization  as  his  task  is  integrated  into  the  project. 
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The  clear-cut  chain  of  command  is  essential  to  the  accomplishment  of  the  higher-order  tasks 
at  the  system/subsystem  level  and  to  the  maintenance  of  accountability  among  those  work¬ 
ing  on  the  project. 

Obviously,  many  tasks  are  contingent  upon  the  completion  of  other  tasks.  The  rela¬ 
tionships  between  tasks  are  important  to  two  essential  management  functions:  tracking  pro¬ 
gress  and  risk  management.  When  estimating  the  time  required  to  complete  a  task,  it  is 
important  to  consider  the  impact  on  other  tasks  if  schedule  must  be  slipped  or  cannot  be 
made  at  all.  If  this  effort  is  included  in  the  initial  planning,  realistic  alternative  plans  can  be 
developed  and  appropriate  allowances  can  be  made  for  schedule  slippages.  A  common  error  in 
milestone  planning  is  to  assume  that  each  task  will  be  completed  in  an  expected  (most  likely) 
time.  In  practice,  each  task  will  be  completed  early  or  late  with  the  average  of  all  tasks  at 
the  expected  time.  But  early  completions  of  a  subtask  frequently  do  not  contribute  to  an 
early  completion  of  the  larger  task,  whereas  a  late  completion  of  a  subtask  may  make  late 
completion  of  the  larger  task  inevitable.  Studies  of  large  numbers  of  projects  in  the  govern¬ 
ment  and  industry  show  an  average  schedule  slippage  of  25%  beyond  the  expected  end  date. 
Careful  milestone  planning  will  compensate  for  inherent  slippages  so  that  the  end  objective 
is  reached  when  promised. 

MANAGEMENT  INFORMATION  SYSTEMS 

Project  management  must  be  cognizant  of  the  progress  and  problems  of  the  tasks  in 
order  to  correct  problems  early  and  to  steer  the  project  to  a  successful  completion.  It  is  im¬ 
possible  to  wholly  replace  personal  contact  between  the  manager  and  the  performers;  how¬ 
ever,  very  large  and  complex  projects  make  adequate  personal  contact  difficult  to  achieve. 
Literally  hundreds  of  management  information  systems  have  been  evolved  to  assist  manag¬ 
ers  in  keeping  track  of  their  projects.  To  be  useful,  a  management  information  system  must 
have  the  following  characteristics: 

The  manager  must  be  familiar  with  it. 

Reporting  personnel  must  understand  it. 

It  must  track  all  the  elements  critical  to  the  particular  project  (and  preferably  no  oth¬ 
ers)  (note:  these  vary  from  project  to  project). 

It  must  be  timely  and  accurate. 

It  must  be  easy  to  implement. 

Management  information  systems  may  run  the  gamut  from  informal  memos  to  highly  struc¬ 
tured  computerized  systems.  From  the  characteristics  above,  it  can  be  inferred  that  the  sys¬ 
tem  is  nothing  more  than  a  method  of  communicating  essential  information.  Often  the 
system  also  provides  a  formal  record  of  progress  among  organizations.  Many  of  the  systems 
are  expensive  to  implement  because  they  require  extensive  changes  to  the  reporting  organi¬ 
zation’s  way  of  doing  business.  Wherever  possible,  the  manager  and  the  reporting  organiza¬ 
tions  should  determine  a  system  by  mutual  agreement.  Also,  ask  only  for  information  which 
can  be  used;  the  purpose  of  an  information  system  is  defeated  by  extraneous  data  which  can 
obscure  real  problem  areas. 

It  is  far  better  to  limit  the  size  of  a  project  so  that  personal  contact  can  be  used  for 
management  information  than  to  have  to  implement  a  management  information  system  in 
the  first  place. 
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FUNDING 


No  acquisition  program  can  even  begin  without  funding.  No  matter  how  badly  the 
Fleet  wants  or  needs  the  product  of  a  good  idea,  the  product  cannot  reach  the  Fleet  unless  a 
sponsor  is  found  for  the  acquisition  program.  Within  the  vast  acquisition  community  bu¬ 
reaucrat,  there  are  a  number  of  pots  of  discretionary  funds.  To  tap  these  funds,  the  project 
must  appeal  to  the  prospective  sponsor  and  must  be  something  for  which  he  can  gain  support 
through  his  normal  budget  channels.  Under  these  conditions,  it  is  possible  to  get  some  Rind¬ 
ing  to  get  started.  However,  discretionaiy  funds  are  only  a  mechanism  to  get  a  small  amount 
of  funding  (usually  less  than  $100k).  The  full  project  funding  must  be  obtained  through  the 
budget  cycle;  the  discretionary  funds  should  normally  be  applied  toward  project  planning  ac¬ 
tivities  leading  to  budgetary  estimates.  The  cycle  of  activities  which  lead  to  the  formation  of 
the  DoD  budget  and  (it  is  hoped)  to  the  approval  of  project  funds  is  a  long  one  fraught  with 
pitfalls;  the  project  manager  should  be  aware  of  these  activities  and  plan  his  tasks  to  deliver 
the  right  information  to  the  right  people  at  the  right  time.  The  budget  cycle  is  formalized  in 
the  Planning,  Programming,  and  Budgeting  System  (PPBS).  Virtually  all  planning  in  the  bu¬ 
reaucratic  structure  above  the  project  manager  is  oriented  toward  the  budget.  Programming 
is  deciding  which  budget  plans  will  be  put  into  the  budget  submission.  Budgeting  is  the  dis¬ 
pensing  of  approved  funds  and  reprogramming  those  funds  as  necessaiy  to  cover  budget 
overruns.  Each  level  in  the  chain  of  command  has  some  authority  in  the  review  and 
reprogramming  of  funds;  project  funding  is  susceptible  to  alterations  at  each  level.  The 
higher  review  levels  receive  only  abbreviated  information  about  the  budget  elements,  so 
changes  at  high  levels  are  frequently  more  arbitraiy  and  not  within  the  influence  of  the  pro¬ 
ject  manager.  On  the  other  hand,  the  levels  near  the  sponsor  can  be  influenced.  The  project 
manager  can  greatly  enhance  the  project  funding  prospects  by: 

•  developing  strong  project  justifications 

•  marshaling  user  support 

•  selecting  a  strong  sponsor 

A  project  may  come  under  review  for  potential  cuts  as  often  as  every  3  weeks,  so  a  commit¬ 
ted  sponsor  is  critical.  Equally  important  is  the  timing  of  proposals  and  justifications;  figure 
III-l  shows  the  PPBS  cycle  with  the  points  where  action  may  be  indicated.  For  a  more  de¬ 
tailed  explanation  of  the  PPBS,  see  DoD  Instruction  7045.7,  SECNAV  Instruction  5000.16E, 
and  the  Navy  Programming  Manual. 

When  making  funding  estimates,  characterize  the  estimates  in  accordance  with  OP- 
NAV  Instruction  7040.5;  for  convenience  the  appropriate  categories  are  listed  in  table  III-l. 
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Figure  IIM.  Planning,  programming,  and  budgeting  system  (PPBS)  cycle. 
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Table  HI-1.  Cost  estimate  categories. 

Category  Description 

A  Contract  for  itemfe)  under  consideration  in  existence.  Cost  estimate 

based  on,  included  in,  or  modifying  existing  contract.  In  operating 
accounts,  estimate  based  on  firm  pay  rates  or  firm  operating  pro¬ 
gram  costs  with  requirements  defined  in  detail. 

B  Request  for  proposal  (RFP)  for  item(s)  under  consideration  in  hand. 

Cost  data  based  on  Navy’s  evaluation  of  the  RFP.  In  operating 
accounts,  estimate  based  on  approved  budgeted  pay  rates  or  operat¬ 
ing  program  costs  with  average  requirement  accurately  defined. 

C  Lowest  budget  quality  estimate.  Based  on  engineering  analyses  of 

detailed  characteristics  of  the  item(s)  under  consideration.  In  oper¬ 
ating  accounts,  estimate  based  on  proposed  budget  pay  rates  or 
operating  program  costs  with  requirements  defined  in  total. 

D  Estimate  based  on  technical  feasibility  studies  and/or  extrapolation 

from  higher  quality  estimates  for  similar  item(s).  In  operating 
accounts,  pay  rates  or  operating  program  costs  should  be  at  least 
category  C  but  operating  requirements  are  defined  only  to  +  or 
- 10%  order  of  magnitude. 

F  Ballpark  estimate.  Proposal  required  further  definition  or  further 

study  to  refine  costs  where  a  higher  category,  save  for  one  or  two 
requirements  thereof,  would  bo  justified;  this  should  be  spelled  out. 


When  formulating  justification,  emphasize  the  project  features  which  are  most  rele¬ 
vant  to  the  interests  of  the  reviewers  and  describe  them  in  terms  appropriate  for  the  budget 
category  of  the  funds  being  made  available  to  the  project.  The  budget  categories  are  de¬ 
scribed  in  table  III-2. 


Table  HI-2.  Navy  budget  categories. 

Research,  Development,  Test  and  Evaluation,  Navy  (RDT&EN) 

6.1  Basic  Research 

6.2  Exploratory  Developraent/Applied  Research 

6.3  Advanced  Development 
6.3a  Technology  originated 

6.3b  Operational  Requirement  originated 

6.4  Engineering  Development 

6.5  Systems  Engineering  Support/Operations  Analysis 

6.6  Operational  System  Development  and  Unallocated 
RDT&E  (to  cover  emergent  requirements) 

Military  Personnel,  Navy  (MPN) 

Operation  and  Maintenance,  Navy  (OMN) 

Procurement  of  Aircraft  and  Missiles,  Navy  (PAMN) 
Shipbuilding  and  Conversion,  Navy  (SCN) 

Other  Procurement,  Navy  (OPN) 

Military  Construction,  Navy  (MCN) 

“Other”  Navy  (Naval  Reserve  and  Marine  Corps) 


Important.  The  budget  cycle  is  at  least  2-3  years  long;  especially  when  large  dollar 
amounts  for  procurements  are  involved;  therefore,  thorough  and  accurate  advanced  planning 
is  essential  to  ensure  that  funding  is  available  to  support  project  activities. 


PROJECT  DOCUMENTATION 


The  number  and  variety  of  documents  required  simply  because  a  project  exists  are 
enough  to  stagger  the  imagination.  Unless  the  project  manager  is  prepared,  the  documenta¬ 
tion  requirements  will  be  overwhelming.  Many  of  the  documents  are  necessaiy  to  obtain 
funding  or  to  gain  various  forms  of  approval.  The  documentation  which  may  be  required  in 
the  course  of  the  project  in  conjunction  with  the  tasks  is  discussed  in  the  Documentation 
section  of  this  guide  (see  chapter  XX).  Despite  all  these  diverse  papers,  there  are  still  docu¬ 
ments  which  will  be  required  in  one  form  or  another  during  the  project.  The  sponsor  may 
author  some  of  these  documents  or  may  task  the  project  to  generate  them.  Table  III-3  sum¬ 
marizes  the  documents,  their  purpose,  and  the  guidance  to  their  preparation  where  avail¬ 
able. 


Usually,  there  is  some  flexibility  in  the  preparation  of  the  documents.  Often  it  is  pos¬ 
sible  to  survey  the  various  required  documents  and  to  write  a  few  paragraphs  which,  with 
some  cutting  and  pasting,  can  satisfy  all  the  requirements.  At  the  same  time,  there  is  no 
point  in  writing  a  series  of  paragraphs  filled  with  pap.  Try  to  make  the  documents  meaning¬ 
ful,  concise,  and  accurate;  then  the  few  times  someone  actually  uses  them,  they  will  work  for 
the  project. 

Important.  Most  program  documents  are  reviewed,  approved,  and  used  by  people  who 
are  busy  with  “fire  drills.”  Therefore,  the  documents  should  be  organized  to  be  read  and  un¬ 
derstood  in  5  minutes  or  so.  Key  issues  and  proposed  resolutions  should  stand  out  from  sup¬ 
porting  information.  Documentation  should  be  information  rich  and  not  obscured  by 
excessive  data  or  verbage. 


Table  HI-8.  Project  documents. 


CategoiyfPepartmeat 

Requirements 

Operational  Requirement  (OR) 
Development  Proposal  (DP) 

Navy  Decision  Coordinating  Paper 
(NDCP) 

Science  &  Technology  (S&T) 
Objectives 


Purpose 

Project  objectives 
Project  approach 
Development  concept 

General  technical 
goals 


Program  Planning 


Reference 


OPNAVINST  5000.42 
OPNAVINST  5000.42 
SECNAVINST  5000.1 

OPNAVINST  5000.42 


Program  Management  Plan  (PMP)  Identifies  project  ob-  See  chapter  XXII 

jectives,  resources, 
constraints,  risks, 
and  management  ap¬ 
proach 
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Table  HI-3.  Project  documents  (continued). 


CatfgnrvfllereiTtment  Purpose 

Program  Planning- 

Test  and  Evaluation  Master  Plan  Identifies  T&E  issues, 
(TEMP)  objectives,  and 

resources 


Integrated  Logistic  Support  Plan 
(ILSP) 


Configuration  Management  Plan 
(CMP) 

Data  Management  Plan 


Security  Plan 

Proposed  Military  Improvement 
(PMI) 

Proposed  Technical  Improvement 
(PTI) 

Procurement  Plans 
Acquisition  Plan 


Miscellaneous 

Research  &  Technology  Work 
Unit  Summary  (DD  1498) 

Quality  Assurance  Plan 


Documents  support 
concepts,  objectives, 
and  constraints;  in¬ 
cludes  plans  and 
shows  relationships 
between  reliability, 
maintainability,  hu¬ 
man  engineering, 
safety,  personnel  & 
training,  provision¬ 
ing,  transportability, 
tasks,  etc. 

Management  approach 
to  maintain  configura¬ 
tion  control 
Procedures  for  pro¬ 
cessing  and  distribut¬ 
ing  project  data 
Project  security  pro¬ 
cedures 

Initiates  installation 
planning 


Major  procurements 
planning  and  request 
for  authority 


Establishes  a  record  of 
the  existence  of  the 
project 

Establishes  project 
procedures  for 
quality  review,  veri¬ 
fication,  and  certi¬ 
fication/acceptance 


Reference 


OPNAVINST  3960.10 


SECNAVINST  5000.39A 
OPNaVINST  5C30.49A 


SECNAVINST  4130.2 


NAVMATINST  4000.15A 
(NAVSUP) 

See  chapter  XXII 

OPNAVINST  4720.2 


Acquisition  Planning 

Guide 

(ASNSdL) 


SECNAVINST  3900.29B 
and  local  instructions 

NAVMAT  4855.1 
(SPAWAR),  MIL-Q- 
9858A,  or  MIL-I-45208 
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Table  1U-3.  Project'documents  (continued). 

Category /Department  Purpose  Reference 

Miscellaneous  (continued) 

Standardization  Plan 


PROGRAM  TAILORING 

If  one  could  give  the  program  objective  to  a  single  engineer,  possessing  the  knowledge 
and  talent  needed,  a  ‘best”  product  would  result.  This  statement  is  worth  repeating  in  order 
to  emphasize  the  necessity  of  keeping  a  project  manageable.  On  the  other  hand,  most  pro¬ 
jects  vastly  exceed  any  single  engineer’s  knowledge,  talent,  or  capability  to  execute  the 
needed  tasks  within  a  specified  time,  so  the  project  team  approach  becomes  the  practical  so¬ 
lution. 


Determines  the  levels  See  chapter  XXII 
of  standardization  to 
be  implemented  and 
parts  selection  of 
criteria 


The  concept  of  doing  just  enough  to  do  a  thorough  and  adequate  job  is  called  tailor¬ 
ing  Most  of  this  guide  is  dedicated  to  tailoring  concepts.  For  instance,  chapter  XX  includes 
procedures  for  tailoring  documentation  requirements.  The  “cover  your  anatomy”  approach  to 
equipment  acquisition  is  extremely  expensive  and  wasteful  of  resources,  and  often  leads  to 
project  failure  by  emphasizing  nonproductive  tasks.  The  techniques  of  tailoring  constitute  a 
balancing  act  between  insufficiency  and  overkill.  The  essence  of  all  the  advice  in  the  chap¬ 
ters  which  follow  is,  “Require  exactly  that  which  is  needed.” 

The  problem  of  program  tailoring  is  that  an  almost  infinite  amount  of  knowledge  is 
required  in  order  to  absolutely  define  what  is  needed.  Gathering  this  knowledge  expends  pro¬ 
ject  resources.  Eventually  a  point  of  diminishing  returns  results  wherein  the  new  knowledge 
gained  requires  more  resource  expenditure  than  what  can  be  saved  in  the  acquisition  of 
equipment.  At  some  point  prior  to  the  point  of  diminishing  returns,  there  will  be  sufficient 
confidence  in  the  information  at  hand  that  decisions  can  be  made  with  an  acceptably  low 
risk. 


The  perception  of  “acceptable  risk”  is  affected  by  technical  difficulty,  resources  ($)  at 
risk,  and  the  level  of  review  interest.  Larger  projects  incur  higher  level  reviews,  but  even 
small  projects  can  receive  high-level  review  interest  because  of  the  potential  military  bene¬ 
fits.  The  higher  levels  of  review  and  greater  resources  at  risk  demand  a  lower  technical  risk 
to  compensate  for  the  higher  political  risks.  A  very  small  project  can  tolerate  higher  risks 
because  a  complete  project  failure  constitutes  a  small  loss.  A  very  large  project  demands  low- 
risk  decisions  because  even  a  partial  failure  constitutes  a  very  large  loss. 

The  information  level  needed  for  very  small  efforts  (the  one-  or  two-man  efforts  on 
the  order  of  $500k  or  less)  can  be  satisfied  by  educated  guesses  in  most  cases.  The  individ- 
ual(s)  executing  the  tasks  should  be  thoroughly  experienced  in  the  critical  tasks  to  be  per¬ 
formed,  such  as  design  or  installation  planning  or  testing.  On  less  critical  tasks,  appropriate 
expert  advice  should  be  located  on  a  free-  or  part-time  basis.  The  task  leader  should  be 
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responsible  for  researching  noncritical  issues  to  his  satisfaction.  The  success  of  the  project 
will  hinge  on  the  selection  of  task  personnel  and  on  the  proper  determination  of  critical  is¬ 
sues. 


At  the  extreme  of  the  very  large  project  (mqjor  program  as  defined  in  DoD  Directive 
5000.1),  supporting  research  and  test  projects  should  be  utilized  to  validate  information  as 
well  as  gather  it.  Full-time  expert  assistance  will  be  utilized  for  a  large  range  of  issues.  The 
size  of  the  program  will  limit  the  number  of  issues  which  can  be  treated  as  noncritical. 

To  illustrate  program  tailoring,  consider  the  issue  of  reliability  as  it  might  be  treated 
by  projects  of  various  sizes.  In  a  veiy  small  project,  reliability  can  probably  be  ignored  as  an 
independent  issue;  good  design  practices  can  be  depended  on  to  yield  sufficient  reliability. 
The  small  project  might  require  a  reliability  prediction  for  support  planning  purposes,  but  a 
cursory  prediction  using  MIL-HDBK-217  can  suffice.  An  intermediate  project  would  require 
a  more  detailed  prediction  and  would  probably  require  a  failure  modes  and  effects  analysis 
(FMEA);  a  reliability  specialist  would  be  employed  to  assist  in  these  tasks.  Also,  interne'' 
ate  projects  may  use  reliability  testing  as  a  portion  of  qualification  tests.  Large  and  very 
large  projects  may  employ  a  full-time  reliability  specialist  to  directly  influence  the  equip¬ 
ment  design  and  to  perform  other  reliability  tasks;  reliability  testing  should  play  a  signifi¬ 
cant  role  in  the  reliability  program  as  a  formal  reliability  growth  technique  may  be  utilized. 
These  brief  descriptions  show  how  a  given  issue  gains  importance  with  project  size  and  man¬ 
agement  response  grows  accordingly.  In  each  case,  it  is  assumed  that  reliability  is  not  a 
critical  issue  in  the  operational  requirement;  if  it  were,  a  more  intensive  effort  would  be 
made  for  each  project  size. 

Table  III-4  describes  the  levels  of  information  which  give  various  degrees  of  confi¬ 
dence  to  the  decision-making  process.  Refer  to  chapter  XIX  for  information  regarding  the 
test  and  evaluation  procedures  which  are  essential  to  high-confidence  decisions.  Figures 
III-2  and  III-3  are  provided  to  help  conceptualize  the  relationship  between  project  size  and 
the  acceptable  levels  of  information  for  project  decisions. 

Require  what  is  needed,  reject  the  unnecessaiy.  Obtain  expert  advice  during  early 
program  planning  to  assist  in  the  tailoring  process. 


Table  in -4.  Levels  of  information  for  decision  making. 


0.  Noneducated  guess 

1.  Educated  guess  by  a  nonexpert 

2.  Educated  guess  by  an  expert 


^  guesses 


3.  Expert  advice 

4.  Research  and  analysis 


} 


information  developed 
from  experience 


5.  Analysis  and  simulation 

6.  Diagnosis  of  prior  experience 

7.  Partial  testing  in  use  (such  as  a  Fleet  Research 
Investigation  or  Development  Assist) 

8.  Full-scale  testing  in  use  (such  as  an  Operational 
Assist  or  OPEVAL) 


validated  information 
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CONFIDENTIAL  LEVEL 


LEVEL  OF  INFORMATION 

Figure  111-2.  Level  of  information  (from  table  111-4)  vs  confidence  and  effort  to  obtain. 


RANGE  OF  PROJECT  ISSUES 


ACQUISITION 

CATEGORY 

REVIEW 

INTEREST 

PROJECT  SIZE 
ROUE  $ 

PRODUCTION  $ 

| 

SECDEF 

>  500M 

>  IB  (VERY  LARGE) 

II 

SECNAV 

>  100M 

>  500M  (LARGE) 

III 

OPNAV 

<  100M 

<  100M  }"NTERME 

IV  A 

SYSCOM 

<  20M 

IV  B 

SYSCOM 

<5M 

<20M\ 

IV  C 

SYSCOM 

<2M 

<  5M  /  (SMALL) 

!V  D 

SYSCOM 

<  500k 

<  2M  (VERY  SMALL) 

Figure  111-3.  Project  size  vs  range  of  project  issues. 
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CHAIN  OF  COMMAND 


The  project  manager  and  systems  engineer  must  respond  to  two  chains  of  command 
—  their  own  administrative  chain  of  command.(NOSC  branch  head,  division  head,  depart¬ 
ment  head,  technical  director.  Commanding  Officer,  Director  of  Navy  Laboratories,  Com¬ 
mander,  Space  &  Naval  Warfare  Systems  Command)  and  the  project  chain  of  command 
(acquisition  manager,  OPNAV  sponsor,  SEVNAC  DoD).  The  acquisition  system  is  governed 
by  law,  rules,  regulations,  directives,  and  instructions;  these  regulatory  constraints,  policies, 
and  procedures  are  implemented  through  the  administrative  chains  of  command  of  each  or¬ 
ganization  participating  in  the  project. 

The  separate  organizations  each  have  their  own  set  of  implementing  instructions, 
which  may  cause  conflicts  between  organizations  participating  in  a  project  and  conflicts  with 
specific  project  goals.  Often  an  organization  will  have  implementing  instructions  which 
define  a  standard  operating  procedure  and  which  require  approval  for  deviations  from  the 
standard.  It  is  important  for  the  project  manager  to  become  familiar  with  the  key  instruc¬ 
tions  of  each  organization  in  the  project  chain  of  command  as  well  as  those  in  the  adminis¬ 
trative  chain  of  command.  When  potential  conflicts  exist,  the  project  manager  should  initiate 
a  preferred  resolution  in  writing  and  obtain  approval  from  each  chain  of  command.  Table 
III-5  provides  a  listing  of  the  instruction  standard  subject  classification  numbers  which 
should  bo  investigated  for  applicability  to  a  project.  When  key  documents  are 
obtained,  check  their  references  to  determine  if  additional  documents  are  required. 

Table  m-5.  Key  instruction  standard  subject  classification  numbers  for  projects. 


SECNAVINST  5210.11C  Standard  Subject  Identification  Codes 


1500 

Training 

4330 

(Warranty  provisions) 

2000 

Telecommunications 

4400-4499 

Supply  Material 

2200-2299 

COMSEC 

4408 

Spare  Parts 

2240 

TEMPEST 

4423 

Provisioning  and  Documentation 

2400-2499 

EM  Spectrum,  esp  2400, 2410, 2420 

4490 

Advanced  Planning 

2450 

EMC 

4700-4799 

Maintenance,  Construction,  Conversion 

3000 

Operations  and  Readiness 

4700 

General 

3080-3099 

Reliability  and  Maintainability 

4720 

Alterations  and  Improvements 

3500 

(Embedded  Software) 

4750 

Upkeep  (PMS) 

3900-3999 

RDT&E 

4790 

Maintenance  Engineering/Lcvel  of  Repair 

3900 

General 

4800-4899 

Production  Preparedness 

3910 

Plans 

4855 

Quality  Assurance/Control 

3960 

T&E 

4858 

System  Effectiveness/Value  Engineering 

3961 

TEMPs 

5000 

General  Standard  Operating  Procedures 

4000 

Logistics  (General) 

5100 

System  Safety 

4105 

ILS 

5200-5299 

Management  Programs  and  Techniques 

4120 

Standardization  (Specifications, 

5200,5234 

Software  Standards 

parts  selection) 

5370 

Standards  of  Conduct,  Ethics 

4130 

Configuration  Management 

5500-55 99 

Security 

4140 

Life-cycle  Cost  Model  Records 

5510 

Information  Security 

4200-4399 

Contracting 

7000 

Financial  Management  (Cost  Estimating) 

4200 

General 

8000 

Ordnance  Material 

4210 

Policies 

9000 

Ship  Design 

4275 

Contract  Clauses  Administration 

13000 

Aeronautical  Material 

13050 

Avionics  Configuration  Management 

13070 

Avionics  Reliability 

13630 

Aeronautical  ATE 
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IV.  CONCEPTUAL  PHASE 


Decisions  made  during  the  conceptual  phase  dedicate  a  signifi¬ 
cant  portion  (typically  70%)  of  a  system’s  total  life-<ycle  costs; 
sometimes  costs  can  be  affected  by  as  much  as  two  orders  of 
magnitude.  In  order  to  achieve  the  most  overall  performance  at 
the  lowest  cost,  the  conceptual  phase  must  consider  the  total 
user  requirements  while  completing  the  normal  tasks  of  the  pro¬ 
ject  phase.  This  chapter  discusses  the  tasks  to  be  performed  and 
ways  to  integrate  the  total  requirement  into  them.  The  manage¬ 
ment  of  technical  risk  is  included  as  a  primary  task. 

INTRODUCTION 


The  conceptual  phase  of  any  program  or  project  is  critical  to  its  ultimate  success  or 
failure.  In  general,  this  phase  is  placed  in  jeopardy  by  the  lack  of  resources  made  available  to 
resolve  the  very  large  uncertainties  bearing  upon  the  feasibility  of  applying  technologies  to 
perform  the  tasks  needed  to  satisfy  the  stated  requirement.  The  problem  thus  posed  is  fur¬ 
ther  exacerbated  by  the  ability  to  defer  unresolved  questions  into  later  program  phases.  On 
the  other  hand,  it  would  require  an  inordinate  amount  of  time  and  expenditure  of  resources 
to  completely  resolve  all  pertinent  issues.  Some  amount  of  risk  must  be  assumed  in  order  to 
satisfy  the  stated  requirements  expeditiously.  Therefore,  the  tasks  of  the  conceptual  phase 
are  twofold: 

To  formulate  feasible  approaches  to  the  problem  solution 

To  assess  and  manage  the  risks  associated  with  each 

These  tasks  are  normally  performed  in  two  closely  associated  steps  known  as  concept  formu¬ 
lation  and  technical  approach  development.  The  first  step  delineates  one  or  more  promising 
concepts;  the  second  integrates  into  the  overall  program  plan,  assesses  risks,  and  provides 
proof  of  feasibility  (or  infeasibility)  for  each  concept  (see  figures  IV-1  and  IV-2). 

A  most  important  element  in  concept  formulation  is  ensuring  that  as  complete  a  set 
of  factors  as  possible  is  used  to  identify  promising  approaches;  these  factors  include,  but  are 
not  limited  to,  the  following: 

•  Functions,  modes,  inputs,  and  outputs  dictated  by  the  requirements 

•  Mission  constraints,  such  as  reaction  time,  probability  of  detection,  and  probabil¬ 
ity  of  error,  preferably  embodied  in  an  effectiveness  model 

•  Support  constraints,  including  minimum  acceptable  values,  for  reliability,  main¬ 
tainability,  availability;  goals  for  operability  and  transportability;  and  safety  re¬ 
quirements 

•  Environmental  constraints  tailored  to  the  worst-case  conditions  under  which  the 
equipments  are  expected  to  operate  (not  necessarily  full  MIL-SPEC) 

•  Cost  constraints  in  both  acquisition  and  support  of  the  equipments,  assessed 
through  an  LCC  model 
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•  Producibility  constraints  arising  from  the  quantities  required  in  the  near  term 
and  far  term 

Ultimately,  the  system  must  be  capable,  supportable,  and  affordable.  The  importance 
of  a  well-defined  operational  requirement  sis  a  baseline  to  this  effort  cannot  be  overstressed. 
The  metho'S  of  defining  requirements  depend  on  the  type  of  requirement.  In  general,  opera¬ 
tional  requirements  will  fall  into  one  of  the  following  categories: 

1.  Replacement  of  an  existing  capability  or  group  of  capabilities 

2.  Replacement  of  existing  capabilities  plus  additional  features  which  enhance  those 
capabilities 

3.  Replacement  of  existing  capabilities  plus  new  capabilities 

4.  Supplanting  of  existing  capabilities  by  new  capabilities 

5.  Entirely  new  capabilities 

A  capability  is  an  ability  to  accomplish  a  sub-mission  (i.e.,  to  detect,  to  communicate,  to 
track);  the  sum  of  capabilities  constitutes  an  ability  to  perform  operational  missions  (i.e., 
ASW,  AAW,  gunfire  support,  convoy  escort).  Requirements  to  replace  an  existing  capability 
(categories  1-3)  can  usually  be  stated  as  deficiencies  to  be  corrected  or  improved  upon  while 
assuming  nondeficient  characteristics  as  extant.  Category  1  requirements  almost  always 
arise  from  obsolescent  equipments  which  are,  or  are  becoming,  difficult  to  maintain,  hard  to 
support,  unreliable  compared  to  what  can  now  be  attained,  and  so  forth;  detailed  informa¬ 
tion  on  the  support  characteristics  and  support  parameters  (MTBF,  MTBCM,  MTTR,  Ao, 
MDT)  should  be  obtained  on  the  existing  equipments  and  used  to  establish  minimum  accept¬ 
able  parameters  and  design  goals  for  the  current  project.  Category  2  requirements  generally 
involve  performance  deficiencies  but  may  also  contain  support  deficiencies;  the  required  per¬ 
formance  characteristics  may  usually  be  generated  through  threat  analysis  and  operations 
analysis,  if  they  are  not  explicitly  stated  in  the  requirements  document.  Categoiy  3  require¬ 
ments  usually  arise  from  expanding  missions  or  where  it  has  become  desirable  to  automate 
previously  manual  functions;  category  3  performance  characteristics  will  tend  to  be  more 
operationally  oriented  than  those  of  category  2,  but  are  similar  in  the  steps  needed  for  their 
derivation.  All  requirements  based  in  existing  capabilities  (categories  1-4)  should 
depend  on  a  detailed  knowledge  of  those  capabilities  and  their  deficiencies  in 
performance  and  supportability  to  derive  the  goals  for  the  new  equipments. 

Requirements  based  on  new  capabilities  to  some  extent  (categories  3-5)  must  have 
definitive  characteristics  stated;  if  not  supplied  in  the  requirements  document,  threat  analy¬ 
sis  and  operations  analysis  will  be  necessary  in  some  degree.  However,  the  best  method  (in 
terms  of  the  accuracy  of  information  obtained)  for  determining  the  required  characteristics 
is  a  Fleet  Investigation  (see  OPNAVINST  3960.10  series).  The  Fleet  investigation  should  be 
instrumented  to  measure  all  parameters  pertinent  to  the  particular  threat  scenario,  so  that 
effective  methods  of  simulation  and  accurate  test  plans  may  be  generated  to  support  the 
proof  of  feasibility  phase. 

As  a  guiding  policy  replacement  capabilities  should  be  at  least  as  good  in  performance 
supportability  as  replaced  capabilities  at  less  total  life  cost;  or  should  supply  improved 
performance/supportability  at  the  same  cost;  or  both.  New  capabilities  should  be  incorpo¬ 
rated  within  well  defined  bounds  for  acquisition  cost,  support  costs,  and  logistics  require¬ 
ments. 
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Ultimately,  the  operational  requirements  must  be  transformed  into  technical  (specifi¬ 
cation)  requirements.  The  following  sections  discuss  the  formulation  of  the  various  technical 
parameters;  however,  a  guiding  force  in  the  ensuing  tradeoffs  must  be  the  criticality  of  the 
system  or  equipments  to  the  operating  organization  (ship,  airwing,  etc.)  which  it  services. 


The  following  definitions  are  provided: 

Vital 

Equipments  which  are  essential  to  the  primaiy  missionfe)  of  the 
organization  or  for  damage  control  or  for  personnel  safety 

Semivital 

Equipments  which  are  essential  to  secondary  missions  or  which  are 
supportive  to  the  primary  missions,  damage  control,  or  personnel 
safety 

Nonvital 

All  other  equipments 

Note  that  a  vital  system  (such  as  uhf  shipboard  communications)  might  be  made  up  of  semi- 
vital  equipments  by  virtue  of  the  inherent  system  redundancy.  Conversely,  an  equipment 
which  is  common  to  a  number  of  semivital  systems  might  be  considered  vital  if  no  sufficient 
backup  is  provided.  These  categories  are  useful  in  establishing; 

The  level  of  risk  which  can  be  acceptably  assumed  without  terminating  the  program 
(see  table  IV-1) 

The  acceptable  limits  for  support  parameters,  especially  mean  downtime  and  maxi¬ 
mum  downtime 

The  priorities  and  proportional  shares  of  funding  and  manpower  resources  which 
can  reasonably  be  allocated  to  the  equipment  in  development,  procurement,  and  sup¬ 
port 

On  the  last  point,  it  is  obvious  that  the  cost-benefit  payofls  must  be  significantly 
higher  for  a  nonvital  or  semivital  equipment  to  justify  equal  consideration  with  a  more  vital 
equipment  competing  for  the  same  resources.  The  determination  of  system  criticality  is  nor¬ 
mally  stated  in  the  requirements  documentation,  although  sometimes  implicitly,  and  must  be 
considered  in  the  program  planning  (see  chapter  III),  as  well  as  in  the  determination  of  sup¬ 
port  parameters  (see  SUPPORT  PARAMETERS  in  this  chapter). 

The  above  considerations  require  a  great  deal  of  information  in  order  to  make  intelli¬ 
gent  decisions  early  in  the  conceptual  phase.  In  large  measure,  the  manager’s  ability  to  con¬ 
trol  risks  and  to  achieve  ultimate  project  success  rests  in  his  knowledge  of  what 
uncertainties  exist  and  his  utilization  of  available  information  to  make  correct  determina¬ 
tions.  The  next  section  deals  with  prospective  sources  of  information. 
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Figure  IV-1.  Concept  formulation. 
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Rout*  IV-2.  Technical  approach  ((or  each  possible  concept). 
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Table  IV-1.  Initial  risk  assessment 


System  Character 

The  major  items  in  the  system 
have  been  previously  developed 
and  used  together.  (Develop¬ 
ment  of  noncritical  ancillary 
items  is  permitted.)  (Minor  mod¬ 
ifications  are  allowed.) 

The  major  items  in  the  system 
have  been  previously  developed 
but  have  not  been  used  together 
before. 


One  or  more  of  the  major  items 
in  the  system  require  develop¬ 
ment. 


Risk 
Assess¬ 
ment  Action 

low  Choose  a  Preferred  Approach  which  seems  most  prom¬ 
ising  to  meet  project  objectives;  several  approaches  may 
be  considered  preferred  if  no  clear-cut  advantages  exist 
for  one  over  the  others. 


mod-  Choose  a  Primary  Approach  (identical  to  Preferred 
erate  Approach  under  low-nsk  conditions)  and  at  least  one 
Secondary  Approach  differing  from  the  Primary  Ap¬ 
proach  in  the  area  of  highest  risk  (most  technical  uncer¬ 
tainty). 

high  Choose  Parallel  Approaches  for  each  item  requiring 
development.  At  least  one  of  the  approaches  should  be 
as  conservative  as  possible  (i.e.,  involving  minimal 
uncertainties). 


GATHERING  INFORMATION 


GENERAL 

Before  putting  pencil  to  drawing  board,  the  engineer  will  want  to  obtain  as  much  in¬ 
formation  as  possible  about  the  technology  available  for  application  to  his  design  problem. 
This  section  is  intended  to  servo  as  a  review  of  sources  of  information  that  the  engineer  can 
use  to  determine  ongoing  and  past  technology  that  applies  to  his  forthcoming  system/equip¬ 
ment  formulation.  The  material  obtained  from  this  search  will  allow  for  valid  design  deci¬ 
sions  as  discussed  in  the  next  sections,  decisions  that,  it  is  hoped,  will  result  in  an  optimal 
product  in  terms  of  cost  and  effectiveness. 

Perhaps  the  most  satisfying  and  assuring  way  to  determine  what  is  really  going  on  is 
to  have  a  mouth-to-mouth  confrontation  with  ther  engineers  currently  or  recently  engaged 
in  the  type  of  work  of  interest.  For  such  a  die  j  to  be  of  greatest  benefit,  however,  the  in¬ 
quirer  should  have  previously  obtained  as  much  background  information  as  possible  and 
some  knowledge  about  the  person  to  whom  he  is  talking  and  that  person’s  work.  For  NOSC 
engineers,  the  Research  Library  has  practically  all  the  sources  of  information  needed  to  pre¬ 
pare  for  meaningful  discussion  and  later  design  decisions.  As  a  general  rule,  every  informa¬ 
tion  gathering  effort  should  begin  with  a  request  for  assistance  from  the  Public  and 
Information  Services  Librarian  to  save  time  and  avoid  trouble. 

The  remainder  of  this  section  covers  the  information  services  of  research  libraries, 
two  outside  sources  of  information,  design  engineering  department  services,  and  the  Military 
Parts  Control  Advisory  Group. 
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RESEARCH  LIBRARY  SERVICES 


Library  services  are  commonly  available  to  personnel  seeking  information  on  avail¬ 
able  technology  and  related  industry  and  government  organizations  and  personnel.  The  fol¬ 
lowing,  all  available  at  NOSC,  are  typical. 

Defense  Documentation  Center  (DDQ 

Visual  Search  Microfilm  File  (VSMF) 

Lockheed  Information  Retrieval  Service  DIALOG 

Systems  Development  Corporation  ORBIT  II 

Western  Research  Application  Center  DATACON 

Army  ECOM  Joint  Electronics  Type  Designations 

Miscellaneous  Publications 

Defense  Documentation  Center  (DDC) 

The  DDC  is  the  Department  of  Defense  central  facility  for  RDT&E  information.  One 
function  of  the  DDC  is  to  acquire,  store,  announce,  retrieve,  Jtd  provide  secondary  distribu¬ 
tion  of  scientific  and  technical  documents  directly  to  registered  users,  including  NOSC  and 
its  personnel.  Quick  service  is  provided  by  the  Defense  RDT&E  On-Line  System  of  the  DDC 
which  connects  the  Library  directly  with  the  DDC  RDT&E  data  banks/Univac  1108  by 
means  of  a  CRT  display  keyboard  in  the  library.  Information  services  available  from  DDC 
are:  Announcement;  Technical  Report  Copy;  Bibliography,  Selective  Dissemination  of  Infor¬ 
mation;  DD  1498  Work  Unit  Information;  DD  1634  data  bank;  IR&D  data  bank;  and  Refer¬ 
ral. 


Announcement  Services  comprise  listings  of  additions  to  DDC’s  technical  report  col¬ 
lection  as  announced  in  the  Commerce  Department’s  Government  Research  Announcements 
and,  for  classified  and  limited-distribution  documents,  DDC’s  Technical  Abstract  Bulletin. 
Both  are  issued  semimonthly  and  provide  descriptive  entries  and  abstracts  for  reports  as 
well  as  indexes.  • 


reports  either  in  hard  copy  or  on  microfiche. 


users,  upon  request,  copies  of  technical 


Bibliography  Service  offers  users  abstracts  of  selected  documents.  In  addition  to  a 
number  of  standard  bibliographies,  DDC  will  make  a  computer  search  to  locate  those  reports 
most  pertinent  to  a  user’s  research  project. 

Selective  Dissemination  of  Information  (SDI)  is  an  Automatic  Document  Distribution 
service  which  provides  regular  distribution  of  microfiche  copies  of  newly  acquired  documents 
selected  to  match  a  user's  specific  subject-interest  profile. 

DD  1498  Work  Unit  Information  Service  provides  answers  concerning  the  who,  what, 
when,  where,  how,  costs,  and  other  information  of  DoD-sponsored  research  and  technology 
in  progress.  This  information  includes  the  name  and  phone  number  of  the  person  performing 
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the  work.  The  system  is  designed  to  enable  the  individual  user  to  determine  the  status  of 
research  and  technology  effort  being  performed  in  house  or  under  contract  or  grant;  histori¬ 
cal  information  also  may  be  obtained. 

The  TIT)  1R34  Data  Bank  contains  R&D  program  planning  information  at  project  and 
task-area  levels. 

The  TR&T)  Data  Rank  contains  proprietary  information  on  defense-related  in-house 
work  from  companies  in  the  DoD-Industry  Independent  Research  and  Development  (IR&D) 
program.  Because  the  information  is  proprietary,  its  use  is  limited  to  those  within  DoD.  The 
type  of  information  provided  is  similar  to  that  of  the  DD  1498  Work  Unit  Information  Sys¬ 
tem.  The  IR&D  data  bank  can  be  used  through  channels  other  than  the  on-line  terminal,  as 
explained  in  DSAM  4185.11,  the  IR&D  User's  Manual. 

nnr.’s  Referral  Service  provides  information  about  DoD-sponsored  specialized 
sources  of  scientific  and  technical  knowledge.  When  information  exceeding  that  contained  in 
DDC  is  required,  this  service  is  used  to  direct  requesters  to  the  National  Referral  Center  for 
Science  and  Technology  and/or  other  potential  sources  of  information. 

Visual  Search  Microfilm  File  (VSMF) 

A  room  is  set  aside  in  NOSC’s  Research  Library  for  use  of  VSMF  equipment  —  racks 
holding  the  16-mm  roll-film  cartridges,  a  microfilm  reader-printer,  and  hard-copy  indexes 
and  user’s  manuals.  Two  main  categories  of  information  are  contained  on  film  for  retrieval 
on  the  reader-printer:  “Product  Information”  and  “Specification  and  Standards.”  The  system 
is  set  up  for  self-use  after  consultation  with  the  Public  and  Information  Services  Librarian. 

Product  Information  Services  consists  of:  Design  Engineering  Service;  Marine  Engi¬ 
neering  Service;  Integrated  Circuit  Parameter  Retrieval  Service;  and  Semiconductor 
Parameter  Retrieval  Service. 

The  Design  Engineering  Servire  includes  manufacturers’  data  sheets  organized/filmed 
so  that  like  items  appear  side  by  side  for  rapid  comparison.  Data  are  indexed  by  product 
description  (more  than  35,000  descriptions)  and  manufacturer’s  name  within  the  following 
sections: 

Electrical 

Electronic 

Fluid  Systems 

Instruments 

Materials  and  Fasteners 

Power  Transmission  and  Hardware 

Production  Equipment  and  Services 

The  Marine  Engineering  Service  consists  of  manufacturers’  data  sheets  on  marine- 
unique  products  and  components,  and  is  organized  and  indexed  in  the  same  manner  as  the 
Design  Engineering  Service.  (NOTE:  Although  the  Library  does  not  at  present  provide  this 
service,  it  is  mentioned  here  because  it  is  available  from  the  VSMF  system  in  the  Design  En¬ 
gineering  Division,  NOSC.) 

The  Integrated  Circuit  Parameter  Retrieval  Service  allows  IC  device  selection  by 
defined  function,  original  circuit  number,  and  manufacturer’s  device  number,  and  the  Semi¬ 
conductor  Parameter  Retrieval  Service  allows  discrete  device  selection  by  electrical  and 
physical  parameters. 
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As  for  the  Specification  and  Standards  Services,  information  is  available  on  militaiy 
specifications,  MS  drawings,  militaiy  handbooks,  and  all  sections  of  the  American  Society 
for  Testing  and  Materials  (ASTM)  standards. 

The  MIL  Specification  Service  contains  current  militaiy  and  federal  specifications 
and  standards,  JANs,  and  QPLs  indexed  by  document  number,  title,  and  product  description 
within  the  following  sections: 

Assemblies 

Electrical 

Instrument 

Mechanical 

Procedures 

General 

Two  supplements  to  these  data  also  are  provided:  “Hot  Specs,”  which  are  new,  late-release 
documents;  and  “Historical,”  those  documents  which  have  been  superseded  by  documents  in 
the  current  file. 

The  MS  Drawing  Service  contains  MS,  AN,  AND,  USAF,  NASA,  and  NAS  drawings; 
MIL-D-1000  and  associated  documents;  and  Cataloging  Handbooks  H4-1  and  H4-2;  with 
drawings  of  like  items  filmed  side  by  side  for  easy  comparison.  Indexing  is  by  document 
number,  title,  and  product  description. 

The  Military  Handbooks  Service  contains  all  such  documents,  indexed  by  document 
number  and  title. 

The  American  Society  for  Testing  and  Materials  Service  contains  all  ASTM  stan¬ 
dards,  indexed  by  document  number  and  subject,  in  the  following  sections: 

Metals 

Construction 

Paint 

Petroleum 

Plastics 

Textiles 

Rubber  and  Electric  Insulating  Materials 
General  Test  Methods 
Miscellaneous 

All  VSMF  Services  are  updated  frequently  and  regularly  to  provide  current  informa¬ 
tion.  In  addition,  VSMF  provides  “Extension  99”  —  telephone  data  assistance.  Subscribers 
may  use  a  direct  telephone  link  to  a  staff  of  data  specialists  at  VSMF  headquarters  near  Den¬ 
ver  for  assistance  in  making  a  search,  for  information  beyond  that  available,  etc. 

Lockheed  Information  Retrievai  Service  Dialog 

Five  Lockheed  DIALOG  files  are  available  at  the  Research  Libraiy  which  are  of  par¬ 
ticular  interest  to  Center  personnel. 


IV-9 


NTIS  (National  Technical  Information  Service).  Contains  about  400,000  Govem- 
ment-fimded  unclassified  technical  reports. 

COMPENDEX  (Engineering  Index).  Has  approximately  275,000  citations  in  all 
fields  of  engineering. 

INSPEC  (Institution  of  Electrical  Engineers).  Has  450,000  abstracts  in  areas  of 
electrical  and  electronic,  computer  and  control,  and  physics. 

CHEMCON  (Chemical  Condensates).  Has  over  a  million  entries  in  all  fields  of 
chemistry. 

ERIC  (Educational  Resources  Information  Center).  All  fields  of  educational  research 
are  covered  by  185,000  citations. 

Systems  Development  Corporation  ORBIT  II 

Two  ORBIT  II  files  may  be  of  value.  GEO-REF  (American  Geological  Institute)  con¬ 
tains  about  4  million  items  concerning  work  in  geology,  mineralogy,  oceanography,  geophys¬ 
ics,  geochemistry,  and  geomorphology.  And  SSIE  (Smithsonian  Science  Information 
Exchange)  has  about  170,000  entries  concerning  scientific  research  projects  that  are  cur¬ 
rently  funded  by  federal  agencies  and  state  and  local  governments,  universities,  colleges,  and 
private  foundations. 

Western  Research  Applications  Center  DATACON 

DATACON  accesses  all  NASA  and  NASA-sponsored  research  reports.  (The  current 
capabilities  of  the  three  preceding  data  files  —  DIALOG,  ORBIT  II,  and  DATACON  — 
include  stato-of-the-art  reviews  on  a  given  subject;  single  author,  subject,  or  contract  inquir¬ 
ies;  retrospective  bibliographies  on  a  subject  or  concept;  and  currently  funded  project 
searches.). 

Army  ECOM  Joint  Electronic  Type  Designations 

The  Research  Library  has  16-mm  film  cartridges,  prepared  and  maintained  by  the 
Army  Electronics  Command  (ECOM),  containing  data  on  the  parameters  of  all  nomencla¬ 
ture^  equipments  in  the  DoD  inventory.  The  cartridges  are  usable  on  the  VSMF  reader- 
printer. 

Commerce  Business  Daily 

Issues  of  the  Commerce  Business  Daily  are  located  on  the  periodical  shelves  of  the 
Research  Library.  Clues  and  leads  to  ongoing  technology  of  interest,  and  government  and 
industry  participators,  may  be  found  in  the  sections  on  procurement  invitations  for  services 
and  supplies,  contract  awards,  and  the  solicitation  of  research  and  development  sources. 

Miscellaneous  Publications 

The  annual  US  Government  Manual  and  the  Navy  Technical  Facility  Register  (NAV- 
MAT  P3999-1)  contain  information  on  Government  agencies  and  personnel,  the  latter  being 
particularly  useful  for  looking  up  Navy  R&D  centers  to  determine  facilities,  personnel 


responsibilities,  whom  to  contact,  etc.  The  Public  and  Information  Services  Librarian  also 
can  direct  the  investigator  to  other  documents  which  may  be  helpful,  such  as  specialty  peri¬ 
odicals,  handbooks,  and  annual  buyers’  guides;  society  periodicals  and  handbooks;  and  ven¬ 
dors’  catalogs. 

GOVERNMENT-INDUSTRY  DATA  EXCHANGE  PROGRAM  (GIDEP) 


The  GIDEP  is  jointly  sponsored  by  the  Army,  Navy,  Air  Force,  Canadian  Military 
Electronics  Standards  Agency  (CAMESA),  NASA,  FAA,  DSA,  SBA,  and  other  federal  depart¬ 
ments  and  agencies.  GIDEP  provides  automatic  interchange  of  technical  data  related  to 
parts,  components,  and  materials  utilized  in  military  systems.  The  data  are  primarily  results 
from  environmental  testing  conducted  by  participants  who  are  engaged  in  design,  develop¬ 
ment,  and  production  of  military  and  aerospace  equipment.  Service  is  available  without 
charge  to  users.  Information  on  GIDEP  may  be  obtained  from; 

Officer  in  Charge  (Code  862) 

Fleet  Missile  Systems  Analysis  &  Evaluation  Group 
Corona,  California  91720 

Phone  (714)  736-4677  (Autovon  933-4677) 

DoD  INFORMATION  ANALYSIS  CENTERS 

The  Defense  Department  supports  18  centers  for  analysis  of  scientific  and  technical 
information.  Each  center  gathers  information  in  its  spocial  area  of  interest;  reviews,  ana¬ 
lyzes,  evaluates,  synthesizes,  condenses,  and  summarizes  the  information;  and  provides  it  to 
individual  users.  The  centers  produce  critical  reviews,  state-of-the-art  monographs,  data 
compilations,  and  substantive  responses  to  queries.  A  charge  is  made  for  services  to  both 
government  and  contractor  users.  Information  on  the  particular  center  most  likely  to  bo  able 
to  provide  the  desired  information  may  be  obtained  from  NOSC’s  Public  and  Information 
Services  Librarian. 

DESIGN  ENGINEERING  DIVISION  SERVICES 

The  Design  Engineering  Division  at  NOSC  and  similar  organizations  at  other  Com¬ 
mands  provide  a  number  of  services  to  assist  the  project  manager;  these  include: 

Searches  of  the  Navy's  Maintenance  Data  Collection  System  (MDCS).  MDCS  is  use¬ 
ful  in  analyzing  current  equipment  performance  and  reliability  and  can  help  estab¬ 
lish  design-to-cost  parameters. 

Assistance  in  project  planning  and  especially  in  developing  tailored  specifications, 
tailored  environmental  testing  criteria,  tailored  support  methodologies,  selection 
and  screening  criteria  for  existing  commercial  equipments,  and  life-cycle  cost  esti¬ 
mates. 

Installation  planning. 

Assistance  in  parts  selection  especially  using  military  specifications  and  the 
National  Stock  System. 
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The  Design  Engineering  Division  also  performs  packaging  design,  provides  drafting 
services,  and  coordinates  printed  circuit  layout  and  fabrication. 

At  NOSC,  the  Design  Engineering  Division  also  provides  the  VSMF  service. 

Military  Parts  Control  Advisory  Group  (MPCAG) 

The  MPCAG  provides  free  parts  selection  advice  supported  by  an  extensive  staff  and 
computer  facilities.  Telephone  numbers  and  addresses  to  obtain  assistance  are  listed  in  MIL- 
STD-965. 


PERFORMANCE  PARAMETERS 


There  are  two  aspects  to  the  specification  of  performance  —  the  degree  of  functional 
proficiency  to  be  achieved  and  the  severity  of  the  environment.  Traditionally,  the  specifica¬ 
tion  approach  has  been  to  obtain  as  much  performance  as  possible  and  to  assume  the  envi¬ 
ronments  called  out  in  militaiy  standards  and  specifications;  this  approach  drives  program 
costs  upward  and  creates  unnecessary  technical  risks.  The  approach  recommended  herein  is: 

1.  Specify  only  those  functions,  modes,  and  characteristics  completely  which  are  nec¬ 
essary  to  meet  the  operational  requirements. 

2.  Specify  environments  which  realistically  reflect  the  actual  usage  environments. 

Military  specifications  normally  reflect  a  “worst  case”  environment.  This  is  composed 
of  the  most  extreme  cases  of  a  multitude  of  environments  which  are  usually  grossly  in  excess 
of  the  extremes  normally  encountered  in  any  one  usage  environment.  Table  IV-2  summarizes 
the  environmental  parameters  which  should  be  considered;  appendix  A  provides  more  de¬ 
tailed  environmental  criteria.  Test  and  evaluation  planning  must  be  structured  with  the 
specified  environment  as  a  baseline  (see  chapter  XIX). 

Table  IV-2.  Equipment  environmental  tests  and  requirements. 

GENERAL  REQUIREMENTS 

Designator  Eaviroameat 

1.  Temperature  (operating  and  nonoperating) 

2.  Temperature-altitude 

3.  Humidity 

4.  Thermal  shock 

5.  Vibration 

6.  High-impact  shock 

7.  Transport  shock 

8.  Repetitive  shock 

9.  EMC  (interference  and  susceptibility) 

10.  EMP 

11.  Electrical  transients  (voltage  and  frequency/long  term  and  short  term) 

12.  Lightning 

13.  Magnetic  field 

14.  Acoustic  noise  (airborne  and  structureborne) 
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Table  IV-2.  Equipment  environmental  tests  and  requirements  (continued). 
GENERAL  REQUIREMENTS 
Designator  Environment 

15.  Inclination 

16.  Radiation 

17.  Nuclear  air  blast 

18.  Gun  blast 

19.  Wind 

20.  Icing  with  wind 

21.  Rain  and  snow,  snow  loading 

22.  Sunshine 

23.  Degree  of  enclosure 

24.  Dust 

25.  Salt  fog,  spray,  solution 

26.  Damaging  (corrosive)  atmospheres 

27.  Explosive  atmosphere 

28.  Fungus 

29.  Maintainability/bench  handling 

30  Malysisf  (burn'in’  confldence.  indexing,  accelerated  life,  failure-mode 

S1- 

32.  On/off  cycling 

33.  Acoustic  susceptibility  (in  higb-noise  environments) 

34.  Water  impact/hydrostatic  pressure 

36  Drop  tesf01"  exp*os’on  ^or  hull-mounted  equipments  only) 

37.  Equipment  special  environments 


Category 

Vital  equipments 

Semivital  equipments 

Nonvital  equipments 
Exposed  equipments 

fijinlf nr ruJ  svr*i tin m 


SPECIAL  REQUIREMENT  CATEGORIES 


Environments 


10,16,17 

6 

6 


V 

_-r. —  12,18,19,20,21,22,23,24 

sheltered  equipments  23 

Standard  requirements  1,3,5,6,9,11,14,29,30,31,32,37 


Notes 

16  and  17  for  exposed  equipments 
operating 

nonoperating  (normal  operating 
before  and  alter  shock) 
safety  criteria 
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Table  IV-2.  Equipment  environmental  teste  and  requirements  (continued). 
APPLICATION  REQUIREMENTS' 


Application 

Environments 

General  Equip¬ 
ment  Snec 

Test  Soec 

Notes 

Shipboard 

8,13,15,25,26, 

27,33,34,35 

MIL-E-16400 

MIL-E-16400 

for  (5)  MIL-STD-167 
for  (6)  MIL-S-901 
Environmental  Interfaces 
MIL-STD-1399 
for  (15)  per  MIL-E-16400 
except  30*  (operating)  and 
60*  (without  damage  or 
spillage)  for  submarines 

Shore (fixed) 

33 

MIL-E-16400 

MIL-E-16400 

for  (5)  .5g  to  50  Hz 

for  (6)  MIL-S-901  may  be 

waived 

Shore  (mobile, 
transportable, 
vehicular) 

2,4,7,15 

M1L-E-4158 

MIL-STD-810 

MIL-STD-169 

MIL-STD-170 

MIL-STD-210 

MIL-STD-1474 

MIL-D-13570 

Airborne 

2,4,7,26,27,28 

MIL-E-5400  MIL-STD-810 

Prop,  Jet,  and  MIL-T-5422  (Navy) 

Helo  Aircraft  Gen  Equip  Spec 

M1L-E-8189 

Missiles,  Boosters, 
and  Allied  Vehicles 

MIL-E-11991 

Guided  Missiles 

Portable 

36  plus  applica¬ 
ble  general  ap¬ 
plication 

Same  as  general 
application 

Same  as  general 
application  plus 
detail  spec. 

Space 

Test  equip¬ 
ment 

2.4.7.8.16.33 

7.24.25.27.28.33 

MIL-STD-1540 

MIL-T-28800 

MIL-STD-1540 

MIL-STD-810 

for  (9)  MIL-STD-1541 
equipments  are  all 
considered  vital 

'Environmental  requirements  should  consider  standard  sheltered  or  exposed,  and  vital- 
semivital-nonvital  requirements  in  addition  to  those  listed. 
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Table  IV-2,  Equipment  environmental  tests  and  requirements  (continued). 
APPLICATION  REQUIREMENTS'  (continued) 


General  Equip- 


Application 

Environments 

ment  Spec 

Teat  Spec 

Notes 

Amphibious 

Shipboard  and  Shore 
(mobile)  combined 

Torpedoes 

22,24,25,28,34 

M1L-T-18404 

MIL-T-18404 

(37)  includes  acceleration 

Shipboard 
fire  control 

2,8,13,15,25, 

26,27,33 

MIL-F-18870 

MIL-T-18870 

Same  as  Shipboard  except  as 
by  MIL-T-18870.  (2)  is  non¬ 
operating  transportation 
test. 

'Environmental  requirements  should  consider  standard  sheltered  or  exposed,  and  vital- 
scmivital-nonvital  requirements  in  addition  to  those  listed. 

In  order  to  establish  those  characteristics  which  can  be  considered  “necessary,”  the 
following  definitions  are  useful: 

Essential  Those  characteristics  dictated  by  the  system  mission 

Less  than  essential  Characteristics  which  contribute  usefully  toward  the  system 
mission 

Nonessentiai  Everything  clso 

Necessaiy  characteristics  can  now  be  defined  as  all  essential  characteristics  plus  only 
those  less-than-essential  characteristics  the  provision  of  which  will  not  complicate  operation 
or  maintenance  and  the  utility  of  which  outweighs  the  cost  of  providing  them.  When  in 
doubt,  leave  it  out!  Some  characteristics  may  be  essential  to  one  technical  approach  and  non- 
essential  to  another;  they  are  specified  only  when  the  technical  approach  is  specified.  System 
specifications  generally  do  not  specify  a  technical  approach  unless  only  one  approach  is 
clearly  acceptable;  equipment  specifications  may  direct  a  particular  technical  approach,  espe¬ 
cially  when  multiple  approaches  are  pursued  in  parallel. 

In  determining  the  specification  of  a  characteristic,  quantity  must  be  considered  as 
well  as  quality.  ‘The  receiver  must  have  high  sensitivity”  —  how  much  is  high?  Most 
specified  parameters  fall  into  a  gray  region  when  a  value  is  not  known.  Several  techniques 
are  available  which  can  be  applied  to  resolve  parametric  unknowns: 

Figure  of  merit 

System  effectiveness  model 

Operations  analysis 

Value  engineering/cost-benefit  analysis 

Laboratory  testing 

Exploratory  development 

Analysis  of  similar  equipment 
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A  system  effectiveness  model  is  useful,  since  it  incorporates  performance  parameters, 
thus  allowing  tradeoffs  between  performance  and  support.  System  effectiveness  is  defined  as 
a  product  of  availability,  dependability,  capability,  and  utilization.  Availability  and  depend¬ 
ability  are  support  parameters  and  are  discussed  further  in  the  next  section.  Utilization  is 
an  adjustment  or  degradation  factor  employed  in  the  event  that  the  system  occasionally  is 
used  under  conditions  of  stress  greater  than  the  design  environment;  otherwise,  it  is  set  to  1. 
Capability  is  the  probability  of  a  successful  mission,  given  that  the  system  is  operated  within 
specifications  and  that  no  failures  occur.  Capability  is  a  routinely  applied  concept  in  ord¬ 
nance  systems  where  it  is  the  product  of  the  probability  of  hit  and  the  probability  of  kill 
given  hit. 

Figure  of  merit  (FOM)  is  a  nonprobabilistic  measure  of  capability  which  can  be  read¬ 
ily  developed  for  most  types  of  electronics  and  converted  to  Capability  (C)  through  the  fol¬ 
lowing  formula: 


FOMjmeasured )  -L 
FOM  (max.  specified )  -  L 

FOM  can  be  established  by  determining  which  parameters  directly  interplay  in  the 
system  performance  and  determining  a  common  unit  of  measure.  The  unit  of  measure  may 
be  simple  (e.g.,  dB)  or  complex  (bits/ms-dB).  Environmental  factors  (such  as  atmospheric 
noise)  and  human  factors  (such  as  operator  proficiency)  may  be  assigned  weights  and 
included  in  the  computation.  Analysis  supported  by  laboratory  tests  is  usually  sufficient  to 
establish  a  maximum  predicted  FOM  (FOM  (specified))  which  ensures  satisfactory  system 
operation  and  a  limiting  figure  (I.)  below  which  the  system  will  not  operate.  The  capability 
figure  established  by  utilizing  a  figure  of  merit  derivation  is  not  necessarily  accurate,  but  it 
is  a  sufficiently  useful  concept  to  warrant  consideration.  System  effectiveness  can  be  con¬ 
verted  to  cost-effectiveness  by  dividing  by  the  total  life  cost. 

The  other  measure  of  capability  is  utility.  A  utility  curve  plot*  e  functional  contribu¬ 
tion  factor  (utility)  against  a  specification  factor.  The  utility  is  scaleu  from  0  to  1.  Specifica¬ 
tion  values  below  the  minimum  acceptable  performance  are  assigned  a  utility  of  0;  the 
minimum  acceptable  performance  is  assigned  a  utility  derived  by  analysis  of  the  specification 
factor’s  proportional  contribution  to  total  system  performance  (this  is  usually  between  0.25 
and  0.65).  Many  utility  factors  can  bo  combined  by  using  weighting  factors  (sum  of  all 
weighting  factors  equals  1).  Each  weighting  factor  represents  the  associated  utility  factor’s 
contribution  to  system  performance,  and  is  obtained  through  analysis  and  testing.  Although 
each  weighting  factor  is  a  function  of  all  utility  values,  a  fixed  nominal  value  is  usually  satis¬ 
factory  in  practice. 


*  n 

C  =  —  £  VtWi  :  where  ^  wi  =  1 

i  i.i 

Utility  curves  are  also  very  useful  in  evaluating  contractor  proposals.  In  this  applica¬ 
tion,  proposals  having  any  utility  value  of  zero  (less  than  minimum  acceptable  performance) 
should  be  rated  as  technically  unacceptable. 
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SUPPORT  PARAMETERS 


The  previous  section  stressed  performance  —  the  capability  of  the  end  product  to  do 
the  desired  job.  Equally  important  to  the  user  is  the  supportability  of  the  end  product. 
Equipments  which  are  not  up  and  operating  are  worse  than  no  equipment  at  all;  the  down 
equipment  is  not  doing  its  intended  job,  no  matter  how  much  capability  was  designed  into  it. 
Furthermore,  nonsupportable  equipments  consume  valuable  resources  in  money  and  man¬ 
power  to  effect  their  repair.  Note  that  of  the  four  factors  which  make  up  system  effective¬ 
ness,  two,  capability  and  utilization,  are  performance  related,  and  two,  availability  and 
dependability,  are  support  related.  Besides  playing  a  mqjor  role  in  system  effectiveness,  sup¬ 
port  parameters  describe  the  support  phase  costs.  Since  support  phase  costs  are  a  major  por¬ 
tion  of  the  total  cost  of  ownership,  supportability  and  affordability  are  incontrovertibly 
intertwined. 

Underlying  availability  and  dependability  are  the  more  familiar  concepts  of  reliabil¬ 
ity,  maintainability,  and  downtime  and  the  measures  associated  with  each.  Since  some  of  the 
terms  involve  technical  shades  of  meaning,  figure  IV-3  has  been  provided. 

Traditionally,  reliability  and  maintainability  have  been  the  primary  factors  con¬ 
sidered  in  supportability  design.  Both  can  be  stated  as  requirements  for  which  tests  can  be 
constructed.  The  usual  measures  of  reliability  and  maintainability  are  MTBF  and  MTTR, 
respectively,  which  arc  combined  in  inherent  availability  (see  fig.  IV-3,  formula  1).  Inherent 
availability  is  a  specification  mechanism  which  constrains  the  tradeoffs  between  MTBF  and 
MTTR  where  it  is  desired  to  achieve  values  in  excess  of  minimum  specified  values. 

Inherent  availability  cannot  be  used  to  predict  the  equipment  availability  in  use; 
it  is  a  specification  tool  only  and  must  be  treated  as  such. 

MTBF  and  MTTR  aro  often  considered  independent  of  the  operating  environment; 
this  is  a  gross  fallacy.  Reliability  depends  on  two  mryor  factors  —  (1)  the  selection  of  reliable 
parts  (those  which  are  produced  by  mature  processes  under  established  quality  control)  and 
(2)  the  use  of  reliable  design.  Performance  must  influence  design  complexity,  parts  counts, 
and  other  factors  affecting  reliability;  however,  the  reliable  design  will  always  utilize  parts 
within  their  ratings  even  under  specified  stresses  (temperature,  humidity,  electrical  tran¬ 
sients,  vibration,  etc.)  and  will  allow  for  variations  in  part  value  arising  from  the  production 
process  and  from  aging  effects.  Maintainability  depends  on  the  fault  isolation  characteristics, 
accessibility  of  arts,  testability  of  functions,  and  ease  of  control  adjustment  inherent  to  the 
design. 


Since  the  operating  environment  can  usually  be  readily  specified  (or  readily  over- 
specified  through  blanket  reference  of  military  specifications),  the  conscientious  engineer 
can  formulate  reliable  designs  (i.e.,  within  the  stato-of-the-art).  The  maintenance  environ¬ 
ment  is  usually  not  widely  recognized  by  either  the  specification  writer  or  the  designer 
responding  to  a  specification.  Nevertheless,  it  is  at  least  as  important  to  specify  and  design 
to  the  maintenance  environment  as  to  the  operating  environment;  otherwise,  the  equipment 
will  experience  limited  availability,  excessive  support  costs,  and  reduced  service  life. 


Definitions 


(Source:  1  LS  Implementation  Guide  -  NAVMAT  P-4000) 


1 .  Availability  (A):  The  probability  that  an  item  will  be  ready  to  perform  within  specification  at 
the  start  of  a  mission. 

2.  Dependability  ID):  The  probability  that  an  Item  will  be  able  to  perform  a  complete  mission 
within  specification  and,  should  a  failure  occur  during  that  time,  can  be  restored  to 
operation  within  specification  within  the  allowed  downtime. 

3.  Reliability,  (R):  The  probability  that  an  Hern  will  be  able  to  perform  a  complete  mission 
within  specification,  given  that  It  Is  operational  at  the  start  of  the  mission.  Alternately,  the 
probability  at  an  Item  will  operate  within  specification  for  a  given  period  of  time. 

4.  Maintainability  (M):  The  probability  that  an  Item  can  be  restored  to  operation  within  speci¬ 
fication  within  a  given  downtime. 

5.  Suppoitability  (S):  The  probability  that  alt  the  elements  necessary  effect  the  repair  of  an 
item  will  be  available  within  a  specified  time. 

6.  MTBF:  Moan  time  between  failures  (or  before  failure). 

7.  MTBUR:  Meantime  between  unscheduled  removals.  (Used  for  replaceable  assemblies.) 

8.  MTBCM  or  MTBCMA:  Mean  time  between  corrective  maintenance  actions. 

9.  MTBP^  Mean  time  between  preventive  maintenance  actions. 

10.  ’•  MTBM:  Mean  time  between  maintenance  actions. 

11.  MI7R:  Mean  time  to  repair. 

1  2.  MCT:  Mean  corrective  maintenance  time. 

13.  MDT:  Mean  downtime. 

14.  Failure:  Any  Item  condition  In  which  the  Item  Is  unable  to  operate  within  specification. 

a.  Relevant  Failure:  A  failure  with  which  the  Item  cannot  perform  Its  specified  mission. 

b.  Nonrelovant  Failure:  A  failure  which  causes  degraded  Item  performance  but  does 
not  prevent  the  hem  from  performing  Its  specified  mission. 

c.  Partial  Failure:  A  failure  of  one  or  more  modes  of  operation  which  are  not  critical  to 
the  Item's  specified  mission. 

15.  Ready  time:  The  time  during  which  the  Item  Is  capable  of  operation  but  not  In  use. 

16.  MCMH:  Mean  corrective  maintenance  man-hours. 

17.  MPMH:  Mean  preventative  maintenance  manhours. 

18.  MMMH:  Mean  maintenance  man-hours. 

Figure  IV-3.  Support  parameter  definitions  and  formulae. 
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Formulae: 


MTBF 

1 .  Aj  (inherent  availability)  —  mtb f + mttr 


2.  A<>  (operational  availability)  =  — MTBM+ ready  time _ 

MTBM+MDT.+ ready  time 

a.  (discounting  preventive  maintenance  by  assuming  scheduling  of  PM  to  not  inter¬ 
fere  with  possible  use) 


b. 


3.  a. 


A<J  _  MTBCM + ready  time 

MTBCM + ready  time  +  MDT 
(assuming  100%  utilization)  Ao  = 

D  =  Rm  +  M  (1-Rm)  (Rm 


MTBCM 
MTBCM  +  MDT 
=  mission  reliability) 


b.  D  =  Rm 


(no  allowable  downtime) 


4. 


5. 


a.  Rm  (mission  reliability)  =  e  7  (T  =  length  of  mission) 

MTBF 

b.  Ro  (operating  reliability)  =  t  T 

MTBCM 

MDT  =  MTTR  +  MDT  (admin)  +  MDT  (parts)  +  MDT  (assistance) 


6.  MDT  (admin)  =  FRD  (fault  recognition  delay)  +  MDT  (test  equipment) 
+  MDT  (documentation)  +  SRT  (supply  requisition  time) 

+C3  (coffee  breaks  and  other  periods  of  technician  nonavailability) 


7.  MTTR  =  Time  required  to  procedural  fault  isolate  to  a  replaceable  assembly 
+time  necessary  to  access  and  replace  the  faulty  assembly 
+time  to  reassemble  and  check  out  the  equipment  Including  realignments  as  neces¬ 
sary. 


8.  MTBM  =  1 

1  i 

MTBCM  +  MTBPM 


Specification  Terms: 

Term  Source 


1.  MTTRo:  MTTR  at  the  organization  level  Table  IV-4 

2.  MTTR| :  MTTR  at  the  Intermediate  level; 

no  more  than  2  x  MTTR. 


Figure  IV-3.  Support  parameter  definitions  and  formulae  (continued). 
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Specification  Terms  (continued): 
Term 


Source 


r 


* 


3.  MTTRd:  MTTR  at  the  depot  level;  no  more 

than  2x  MTTR|  or  as  determined  by  a  life- 
cycle  cost  model/level  of  repair  model 

4.  Annual  preventive  maintenance  time  Table  IV-4 


5.  Cost  per  repair  action  Table  IV-4  —  expected  #  actions 

6.  MTBF  Operational  requirement 


7.  MTBM 

Determined  by  a  life-cycle  model 

8.  A0 

Vital  equipments 

0.99 

Semhrltal  equipments 

0.90 

Nonvltal  equipments 

0.75 

Figure  IV-3.  Support  parameter  definitions  and  formulae  (continued). 


Most  of  the  maintenance  environment  can  be  described  as  a  level  of  maintenance 
capability  at  the  user  organization.  The  remaining  portion  considers  the  capabilities  of  inter¬ 
mediate  and  depot  facilities.  Table  IV-3  describes  levels  of  organization  maintenance  capabil¬ 
ity;  table  IV-4  suggests  some  parameters  which  are  within  these  capabilities.  Equipments 
should  never  be  designed  in  excess  of  the  capabilities  of  the  least-capable  platform;  table  IV-5 
table  IV-5  is  provided  for  guidance.  Wherever  possible,  shipboard  equipments  should  be 
designed  for  level  2  capabilities  with  the  primary  maintenance  load  within  the  abilities  of  an 
E-3  technician  and  with  the  most  difficult  tasks  requiring  only  an  E-4.  The  designer  must  be 
provided  with  constraints  through  the  equipment  specification.  Constraining  requirements 
should  include  MTTR  at  the  organizational,  intermediate,  and  depot  repair  levels  and  a  total 
annual  preventive  maintenance  time  ceiling.  A  target  mean  cost  per  repair  is  also  desirable. 

Notice  that  there  are  two  kinds  of  reliability  (see  fig.  IV-3,  formula  4).  Mission  reli¬ 
ability  is  based  in  the  definition  of  reliability  (fig.  IV-3,  definition  3).  However,  because  of 
redundancy,  multiple  modes  of  operation,  gracefully  degraded  operation,  voting  techniques, 
and  multiple-mission  requirements,  many  equipment  conditions  can  occur  which  require 
maintenance  without  constituting  equipment  failure  (in  the  MIL-STD-781  sense).  Therefore, 
the  specification  should  state  requirements  for  MTBF  and  MTBM.  The  MTBF  requirement 
is  generally  only  important  when  a  specific  mission  time  is  of  interest.  MTBM  is  the  require¬ 
ment  that  affects  the  support  costs  directly. 
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Table  IV-4.  Suggested  parameters  compatible  with  maintenance  capability. 


i 


(6)  These  values  are  for  the  “typical"  system  having  5*6  levels  of  complexity.  Less  complex  systems  should  have  lower  demands;  more  complex  tystems 
may  have  more  costs.  Estimate  the  appropriate  value  using  the  system  life-cycle  cost  model  and  normalizing  to  5  levels  of  complexity  above  piece 


Table  IV-5.  Current  maintenance  capabilities  of  major  ship  classes. 


Capability  Class 

6.  CV,  CVA,  CVN,  CG,  CGN,  LPH,  LHA,  SSBN,  LCC,  AD,  AS,  AR 

5.  .  DD,  DDG.LPD,  SSN,  AGF 

4.  FF,  FFG-1,  LSD,  LST,  AOE,  AOR,  AFS..AE 

3.  SS,  LKA,  LPA,  AO,  AF,  FFG-7,  ASR 

2.  «  PG,PHM,MSO,ATF,ARS,ATS 


Returning  to  availability  and  dependability,  it  can  be  seen  that  reliability  and  main¬ 
tainability  do  play  an  important  role  in  these  factors.  However,  a  missing  term  is  down¬ 
time.  Downtime  consists  of  active  repair  time,  administrative  time,  and  time  awaiting  parts 
and  outside  assistance.  Mean  downtime  is  the  normal  measure  of  downtime  per  maintenance 
action.  The  tightening  DoD  budget  of  recent  years  have  increased  the  impact  of  downtime, 
especially  on  operational  availability  (see  fig.  IV-3,  formula  2).  In  recent  equipment  improve¬ 
ment  programs,  MTBCM  has  been  increased  fivefold  and  MTTR  halved,  but  operational 
availability  has  dropped  from  0.75  to  0.30  because  of  MDT  deterioration.  Referring  to  figure 
IV-3,  formulas  5  and  G,  it  can  be  seen  that  many  of  the  factors  affecting  MDT  are  not 
directly  related  to  design;  nevertheless,  design  features  can  have  a  tremendous  impact  on  the 
various  forms  for  downtime.  In  order  to  provide  a  meaningful  specification  of  operational 
availability,  MDT  must  be  realistically  specified  on  the  basis  of  design  characteristics  and 
the  “facts  of  life”  in  shipboard  routine.  Figure  IV-4  provides  suggested  specification  values 
for  downtime.  It  is  useful  to  specify  operational  availability  while  specifying  only  minimum 
acceptable  values  for  MTBF,  MTTR,  and  MTBM;  this  allows  the  designer  more  latitude 
while  constraining  the  design  more  closely  to  the  desired  end  product.  Tests  can  be  provided 
for  the  reliability  and  maintainability  parameters.  The  remaining  values  are  assigned  in 
accordance  with  the  provision  of  certain  features  (as  in  fig.  IV-A),  and  the  expected  availabil¬ 
ity  is  computed  to  check  conformance  with  the  specification  requirements.  The  formulae  to 
be  used  in  this  computation  must  be  stated  in  the  specification.  Dependability  is  never  speci¬ 
fied;  the  dependability  figure  is  derived  for/from  a  system  effectiveness  model  by  utilizing 
figure  IV-3,  formulas  3b  and  4a. 
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MDT  =  MTTR  +  MDT  (Parts)  +  MDT  (Assist)  +  MDT  (Admin) 


MET.  (Harts) 

N 

MDT(Parts)  =  ^  (port  supply  time  In  houre  for  the  I&  part)  x  XI 

f-i 

(failure  rate  for  tne  |th  part)  x  MTBF  (of  the  equipment) 


MDT  =  MTTR  +  MDT  (Parts)  +  MDT  (Assist)  +  MDT  (Admin) 


MQIPaflS) 


0.02  h  built-in  spares 

0.07  h  battle  spares  (stocked  in  separate  spares  kit) 

0.5  h  preferred  parts  (hlgtujsage  standard  parts  per  MIL-STD-242) 

6.0  h  standard  parts  (per  MIL-SPEC) 

720.0  h  life-cycle  stocked  parts  and  nonstandard  controlled  parts 

4320.0  h  uncontrolled  parts  (not  system  peculiar) 

12,960.0  h  uncontrolled  parts  (system  peculiar) 


[based  on  1/2  hour  for  parts  onboard,  24  hours  for  parts  not-on-board  but  expected  to  be  on-board  shlps- 
Incompany,  30  days  for  parts  not-on-board  but  In-stock,  180  days  for  parts  not-on-board,  not-lnstock 
(commonly  available),  540  days  for  parts  not-on-board,  not-ln-stock  (not  commonly  available)] 


NOTES:  (1 )  A  controlled  part  Is  a  nonstandard  part  which  Is  fully  provisioned;  a  life- 

cycle  stocked  part  Is  a  system-peculiar  part  which  Is  stocked  In  sufficient 
quantity  to-support  the  expected  life  of  the  system  (long-lead-time  Items 
may  be  Included  If  they  are  fully  provisioned).  Uncontrolled  parts  Include 
all  Items  which  are  not  standard  or  controlled. 

(2)  These  values  assume  a  fully  Implemented  logistic  support  plan.  If  any  part 
of  the  logistic  support  Is  not  Implemented,  all  provision  parts  will  appear 
to  be  "uncontrolled  parts." 

Figure  IV-4.  Mean  downtime  (MDT)  specification  values. 
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MPT  fAssistanca) 
Task 


Equipment 

Cateqory 

Organi¬ 

zational 

Repair 

IM  Repair 
(on  board) 

IM  Repair 
(mobile  assist) 

IM  Repair 
{off  ship) 

Depot 

Repair 

Equipment 
Turnaround 
to  on-board 
pod  _  _ 

Equipment 

Turnaround 

toIMpod 

Equipment 
Turnaround 
to  Depot  pod 

Equipment 

Turnaround 

Under 

Warranty 

Vital 

0 

15  min 

96h 

48  h 

14  days 

90  min 

96  h 

14  days 

14-30  days 

Semtatal 

0 

24  h 

96h 

120h 

21  days 

90  min 

48h 

14  days 

14-30  days 

NonvrtaJ  (5) 

0 

120  h 

240  h 

240  h 

90  days 

90  min 

96  h 

14  days 

14-30  days 

Notes’ 

(1) 

(2)  (7) 

(2) 

(3)  (4) 

(3)  (4) 

(5)  (3) 

i 

(3) 

(3) 

(3)  (6) 

Notes: 


(1)  by  definition 

(2)  plus  MDT  (parts)  as  computed 

(3)  MDP  (parts)  equals  zero 

(4)  highly  variable;  depends  on  ship's  schedule 

(5)  highly  variable;  depends  on  organizational  work  loads 

(6)  Depends  on  warranty  provisions 

(7)  Applies  to  aircraft  carriers  and  tenders  only 


Equipment  MDT  (Assistance)  will  be  required  as  follows: 

(1)  Depot  and  equipment  turnaround  as  determined  by  the  equipment  support  con¬ 
cept  do  not  add  In  for  those  Items  for  which  redundancy  Is  provided  within  the 
system. 

(2)  IM  repair  will  be  required  In  direct  proportion  to  the  percentage  of  maintenance 
tasks  which  are  beyond  the  organization’s  level  of  maintenance  capability  (tables 
IV-3  and  IV-5)  assuming  no  special  training,  test  equipment,  or  tools  are  required. 
If  special  training  Is  required,  add  (2  x  number  of  weeks  training  required  -4-  C) 
%,  whore  C  =  1  for  nonvital,  2  tor  semlvital,  and  5  for  vital  equipments.  If  special 
tools  or  test  equipment  are  required;  add  0.15  x  percent  of  tasks  requiring  these 
Hems. 

(3)  These  values  assume  fully  Implemented  logistic  support  and  training  plans.  If 
critical  elements  of  either  plan  are  not  Implemented,  support  will  not  be  available 
In  less  than  180  days. 

Figure  IV-4.  Mean  downtime  (MDT)  specification  values  (continued). 
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MPT  fArlmlnl 

MOT  (Admin)  =  FRD  +  MOT  (TE)  +  SRT+  MDT(Doc)  +  CB 

Delay  Category 

0'”#  ««■«— 

faults  in  category)  0 .03  h  Operator  apparent  faults 

0.25  ti  Primary  mode  faults,  not  operator  appar¬ 
ent 

40  Secondary  mode  faults 

(Faults  detectable  only  by  preventive  maintenance  -  use  0.1  the  Interval 
between  applicable  PMS  actions) 

MDT  (TE)  (downtime  for  tools  and  test  equipment) 

=  l  (category  delay)  x  (%  predicted 

faults  In  category)  n  r 

"  0  Faults  requiring  built-in  test  equipment 

and  tools 

0-8  h  Faults  requiring  standard  tools  and  spe¬ 
cial-purpose  test  equipment 
0.25  h  Faults  requiring  general-purpose  test 
equipment 

0.33  h  Faults  requiring  special  tools 


SRT  (Supply  Requisition  Time) 

=  2  (category  delay)  x  (%  predicted 
faults  In  category) 

MDT  (Doc)  (downtime  for  docu¬ 
mentation) 


Faults  requiring  built-in  or  battle  spares 


0.33  h  All  other  taults 

0.02  h  per  Automated  TM  documentation 

screen 

0.05  h  per  Tech  manuals  built  Into  the  equipment 
AS  pages 
of  tech, 
manual 

+  0.05  h  per  volume  of  tech,  manual  not  built  into  the 
equipment 


CB  (coffee  breafcs  and  other  periods  of  technician  nonavailability) 

53r  E:==;S:S:i 

Figure  IV-4.  Mean  downtime  (MOT)  specification  values  (continued). 
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EQUIPMENT  PARTITIONING 


The  partitioning  of  a  system/equipment  pays  an  increasingly  important  role  as  the 
equipment  design  matures;  this  action  becomes  a  mqjor  issue  in  the  transition  to  production 
(see  chapter  VI).  Ultimately,  the  equipment  partitioning  largely  determines  the  maintain- 
ability/repairability  and  the  availability  of  logistics  support  of  the  end  item.  The  cost  of  sup¬ 
porting  the  equipment  can  be  greatly  influenced  by  the  implementation  of  standardization; 
however,  equipments  which  are  procured  in  large  quantities  will  benefit  less  from  standardi¬ 
zation,  since  equipment-peculiar  items  will  be  procured  in  sufficient  quantities  to  attain 
many  of  the  cost  advantages  of  standard  items.  In  general,  the  following  guidelines  should  be 
followed  in  partitioning  actions. 


Small-Quantity  Items 

1.  Partition  for  maximum  use  of  off-the- 
shelf  items  and  minimize  equipment- 
peculiar  items. 


2.  Partition  for  maximum  internal  stand¬ 
ardization  (i.e.,  minimize  the  number  of 
different  items). 

3.  Utilize  technologies  which  are  suffi¬ 
ciently  mature  for  producibility  and 
sufficiently  state  of  the  art  to  remain 
available  through  the  projected  equipment 
life. 


Large-Quantity  Items 

1.  Partition  for  case  of  maintenance  and 
repair,  providing  functions  which  are 
easily  identified  and  isolated  to  a  replace¬ 
able  assembly. 

2.  Partition  to  obtain  the  most  economic 
level  of  replacement;  use  a  LOR  analysis 
where  appropriate  (see  MIL-STD-1390). 

3.  Points  (2)  and  (3)  for  small-quantity 
items  also  apply. 


Large-quantity  items  are  those  items  for  which  one-time  manufacturing  costs  amor¬ 
tized  over  the  production  quantity  are  small  (under  5%  of  the  unit  cost).  Figure  IV-5  can  be 
used  as  guidance  in  this  determination. 

The  resulting  equipment  partitioning  should  be  integrated  into  the  project  work 
breakdown  structure  to  serve  as  a  basis  of  cost  planning  for  management  control,  and  to 
exercise  tradeoff  models  (as  for  life-cycle  costing),  (See  chapter  III.) 
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INTERMEDIATE  «  LARGE 
INTERMEDIATE  «  SMALL 


WHEN  EACH  ITEM  COST 
WHEN  EACH  ITEM  COST 


IS  (RELATIVELY)  HIGH 
IS  (RELATIVELY)  LOW 


Figure  (V-S.  Levels  of  complexity  as  a  function  of  quantity. 
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V.  VALIDATION  PHASE 


The  validation  phase  serves  as  a  vital  checkpoint  on  decisions  made 
during  the  conceptual  phase  and  provides  for  a  smooth  transition  into 
the  development/procurement  cycle;  This  chapter  cites  the  tasks  neces¬ 
sary  to  achieve  these  ends.  The  tasks  cannot  be  slighted  without 
adversely  affecting  the  equipment  total  life  costs. 

INTRODUCTION 

The  purpose  of  the  conceptual  phase  is  spawn  one  or  more  technical 
approaches  (system  design)  within  the  feasible  limits  of  technology.  The  system 
design  must  be  consistent  with  operational  requirements,  acquisition  strategy(ies), 
and  program  resources.  The  validation  phase  demonstrates  this  consistency.  Each 
technical  approach  embodies  technical  requirements  which  form  the  system  specifica¬ 
tion.  How  does  one  know  that  those  technical  requirements  respond  to  the  opera¬ 
tional  requirements?  The  purpose  of  the  validation  phase  is  to  provide  an  answer  to 
that  question  and  to  provide  tolerances  for  the  technical  parameters.  To  these  ends,  a 
validation  phase  is  mandatory;  however,  its  start  and  finish  may  be  deeply  imbedded 
in  the  conceptual  phase  and  the  transition  to  production  with  no  clearly  defined 
boundary. 

The  first  task  of  the  validation  phase  relates  the  technical  requirements  to  the 
operational  requirements.  There  are  three  weys  this  may  be  accomplished: 

Analysis 

Simulation 

Advanced  development 

This  task  is  often  initiated  during  the  conceptual  phase.  Analysis  is  a  suitable  method 
only  when  the  technical  approach  has  been  proved  by  past  experience  to  adequately 
address  the  operational  requirements.  Simulation  entails  risks  of  uncertainty  which 
increase  exponentially  with  the  complexity  of  that  boing  simulated.  Advanced  devel¬ 
opment  provides  the  greatest  degree  of  certainty.  Validation  by  development  is,  how¬ 
ever,  rather  expensive  and  time  consuming  compared  to  simulation  techniques 
because  an  advanced  development  model  (ADM)  must  be  designed,  built,  and  tested. 
When  the  technologies  being  employed  are  relatively  novel,  development  is  the  only 
satisfactory  method;  thus,  the  atomic  test  program  was  essential  to  the  development 
of  atomic  weapons.  On  the  other  hand,  it  is  foolish  to  build  one  very  complex  item, 
say  an  aircraft  carrier,  to  find  out  whether  that  one  item  is  worth  building  or  to  start 
a  war  to  obtain  a  realistic  wartime  environment.  Usually,  a  balanced  program  utiliz¬ 
ing  both  simulation  and  development  is  necessary. 

For  relatively  simple  items,  the  ADM  and  the  conceptual  phase  feasibility 
demonstration  might  be  combined  for  efficiency.  For  complex  items  required  in  small 
numbers,  the  ADM  can  be  combined  with  the  transition  to  production.  In  all  other 
cases  where  development  is  indicated,  the  ADM  should  be  a  stand-alone  phase.  Fur¬ 
thermore,  if  the  tests  of  the  first  ADM  are  not  satisfactory,  successive  ADMs  may  be 
required  until  the  tests  are  OK. 
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Besides  the  functional/technical  parameters,  it  is  also  necessary  to  validate 
the  suitability  of  the  design  concept  to  the  usage  environment.  This  is  the  second 
task  of  the  validation  phase.  Since  it  is  seldom  possible  to  evaluate  all  the  factors 
which  make  up  “suitability”  in  a  live  environment,  it  is  generally  necessary  to 
validate  the  design  through  analysis  and  simulation.  The  necessary  tasks  are  incorpo¬ 
rated  into  the  Integrated  Logistics  System  (ILS)  tasks  outlined  in  the  1LS  Imple¬ 
mentation  Guide  (NAVMAT  P4000)  and  chapter  X  Formal  design  analyses  may  or 
may  not  be  required  depending  on  the  complexity  of  the  design  and  the  familiarity  of 
the  designer  with  the  usage  environment;  where  the  equipment  is  complex  or  the 
designer  naive  to  the  real-life  user  environment,  specialists  should  be  integrated  into 
the  design  team.  Some  of  the  more  important  tasks  are  embodied  in  the  following  for¬ 
mal  analyses: 

Failure  Modes  and  Effects  Analysis  (FMEA) 

Maintainability  Analysis/Demonstration 
Hazard  Modes  and  Effects  Analysis  (HMEA) 

Human  Engineering  Analysis 
Parts  Selection  Review 

Metrology  Analysis  and  Test  Point/Provisions  Review 

These  tasks  are  really  basic  engineering  practices  which  need  not  be  emphasized 
through  a  formal  program  but  which  cannot  be  overlooked.  They  are  often  repeated 
and  refined  many  times  as  the  design  matures.  Their  early  consideration  is  important 
because  extensive  design  changes  may  be  required  to  correct  the  deficiencies  they 
reveal.  The  earlier  these  changes  are  incorporated,  the  less  they  impact  on  schedule 
and  cost. 

Documentation  is  essential  in  the  validation  phase  in  order  to  provide  a 
baseline  for  the  activities  which  follow.  Engineering  drawings  are  particularly  impor¬ 
tant  to  providing  design  traceability  through  the  changes  which  inevitably  occur; 
change  control  procedures  should  be  implemented  to  ensure  that  the  documentation 
accurately  reflects  the  design. 


VALIDATION  PHASE 


This  is  the  phase  in  which  the  major  program  characteristics  (technical,  cost, 
and  schedule),  through  extensive  analysis  and  hardware  development,  are  validated 
and  is  often  identified  with  advanced  development.  It  is  preferred  to  rely  on  hardware 
development  and  evaluation  rather  than  paper  studies  alone,  since  this  provides  a 
better  definition  of  program  characteristics,  higher  confidence  that  risks  have  been 
resolved  or  minimized,  and  greater  confidence  in  the  ultimate  outcome.  Nevertheless, 
effectiveness  analysis  plays  a  critical  role  since  it  provides  a  structured  vehicle  for 
interrelating  and  evaluating  the  information  developed  as  a  result  of  hardware  devel¬ 
opment,  and  organizes  information  in  a  form  suitablo  for  updating  the  Development 
Concept  Paper  (DCP),  and  for  DSARC  review  leading  to  the  decision  to  enter  full- 
scale  development.  In  an  idealized  case,  this  phase  ends  when  a  brassboard  model  has 
been  demonstrated  successfully. 

The  candidate  and  preferred  $ystem(s)  having  been  defined  in  the  conceptual 
phase,  a  sensitivity  analysis  is  performed  with  the  effectiveness  model.  This  analysis 


will  indicate  the  limiting  parameters  and  priorities  for  each  element  of  the  system 
model,  which  are  expressed  in  terms  of  technical  goals  or  requirements.  The  range  of 
the  permissible  parameters,  properly  related  to  estimates  of  state-of-the-art  capabili¬ 
ties,  establishes  the  degree  of  criticality  of  the  element. 

Along  with  the  sensitivity  analysis,  the  analysis  of  risk  and  uncertainty  per¬ 
formed  during  the  conceptual  phase  should  be  updated,  extended,  and  carried  out  to 
greater  levels  of  detail.  Parameters  which  are  found  to  drive  effectiveness  have  a  high 
risk  consequence  (see  chapter  XXI). 

While  the  critical  system  effectiveness  parameters  can  be  defined  initially  dur¬ 
ing  the  early  conceptual  phases,  thqy  reach  much  greater  definition  in  the  latter 
stages  of  validation  during  the  analysis  effort.  They  provide  the  essential  framework 
for  the  decision  to  enter  into  lull-scale  development. 

The  definition  of  these  parameters  at  this  point  in  the  evolutionary  cycle  of  a 
system  must  be  sharpened  to  the  point  where  the  project  manager  can  demonstrate 
the  following: 

•  The  operational  goals  and  technical  goals  are  in  agreement. 

•  The  technical,  and  hence  operational,  criteria  can  be  met. 

•  The  financial  and  schedule  factors  are  credible. 

•  The  development  risks  are  acceptable. 

•  A  definitive  full-scale  development  contract  can  be  entered  into  with  the 
best-qualified  design  agent  (contractor  or  laboratory). 

To  demonstrate  the  foregoing,  not  only  must  the  parameters  be  clearly  and 
concisely  defined,  but  they  must  be  quantitatively  interrelated.  This  requires  highly 
structured  system  models  in  terms  of  functional  block  diagrams  with  associated  char¬ 
acteristics  values  and  a  completely  structured  system  effectiveness/system  value 
model  with  which  to  analyze  and  evaluate  the  system  models.  The  former  is  an  out¬ 
put  of  validation  efforts  by  the  design  agent.  The  latter,  however,  is  largely  the  result 
of  the  efforts  of  the  system  engineering  staff.  The  success  or  failure  of  validation  will 
be  determined  by  the  degree  of  completeness  of  the  model  and  the  degree  to  which  its 
structuring  conforms  to  the  real-world  situation. 

If  the  system  effectiveness/system  value  model  does  approximate  reality  suc¬ 
cessfully,  the  parameters  can  be  interrelated,  and  the  exercise  of  the  model  for  each  of 
the  competing  systems  produced  by  the  design  agent(s)  provides  a  framework  for 
source  selection  and  demonstrates  the  validity  of  entering  into  full-scale  development, 
continuing  with  further  definition  or  advanced  development  effort,  or  abandoning  the 
project. 

In  addition  to  its  use  as  a  decision-making  tool,  the  model  also  serves  another 
function  during  this  period.  The  sharply  defined  critical  effectiveness  parameters  pro¬ 
vide  the  checklist  for  completing  the  specification  for  full-scale  development.  This  is 
particularly  important  in  that  one  of  the  principal  objectives  of  the  validation  process 
is  to  assure  that  a  complete  and  unambiguous  specification  is  developed  for  the  full- 
scale  development  effort. 


PLANNING  FOR  FULL-SCALE  DEVELOPMENT 


Through  the  process  of  validation,  the  project  manager  has  been  establishing 
a  frame  of  reference  to  define  the  system,  its  technical  goals  and  criteria,  and  the 
measures  by  which  its  effectiveness  in  terms  of  its  mission  life  costs  can  be  evaluated. 
Having  established  this  frame  of  reference,  the  project  manager  plans  the  product 
assurance  for  achieving  an  effective  system. 

During  full-scale  development,  the  weapon  system  [including  all  the  items  nec¬ 
essary  for  its  support  (training  equipment,  maintenance  equipment,  handbooks  for 
operation  and  maintenance,  etc.)]  is  designed,  fabricated,  and  tested.  The  intended 
output  is  a  “hardware  model"  whose  effectiveness  has  been  proved  experimentally 
together  with  the  documentation  needed  for  inventory  use.  An  essential  activity  of 
the  development  phase  is  test  and  evaluation,  both  that  conducted  by  contractors  and 
that  conducted  by  the  Service.  Documentation  of  the  full-scale  development  phase, 
including  the  results  of  effectiveness  analysis,  provides  the  basis  for  updating  the 
DCP  and  convening  DSARC  leading  to  initiation  of  production/deployment. 

The  ultimate  evaluation  of  the  full-scale  development  phase  occurs  during  the 
test  and  evaluation  of  the  developed  system.  If  the  system  model  and  system  effective¬ 
ness  analytic  model  are  valid  and  adequately  defined,  the  system  should  meet  its  test 
and  evaluation  successfully,  and  the  project  manager  will  have  been  successful. 

If  the  system  is  not  satisfactory,  the  models  have  yet  another  function.  The 
data  accumulated  during  T&E  should  be  inserted  into  the  models.  The  models  should 
then  be  exercised  and  the  results  analyzed  to  identify  problem  areas.  Theso  should 
then  be  recorded  and  made  available  to  other  project  managers  to  assist  them  in 
avoiding  similar  errors.  At  the  same  time,  a  closed-loop  management  system  should 
be  implemented  to  correct  the  problems. 

If  the  project  is  to  be  continued,  whether  or  not  tho  T&E  is  successful,  the 
T&E  data  are  inserted  into  the  models  to  sharpen  further  the  definition  of  technical 
goals  and  criteria  and  to  validate  the  data  for  the  production  baseline  and  production 
specification.  Here,  again,  the  models  serve  to  guide  the  effort  and  to  assure  the  pro¬ 
ject  manager  that  the  baseline  (specification)  is  complete  and  dofined  as  sharply  as 
tho  aggregate  experience  will  permit.  This  is  a  necessary  exercise,  whether  or  not  the 
R&D  contractor  is  also  the  initial  production  contractor. 


INTEGRATED  LOGISTIC  SUPPORT  (ILS) 
VERIFICATION  AND  DEMONSTRATION 


The  program  requirements  for  Integrated  Logistic  Support  of  NAVELEX 
procurements  are  contained  in  MIL-STD-I3G9.  ILS  verification  and  demonstration 
are  covered  in  paragraph  4.8  and  appendix  C  of  that  standard.  There  are  three  stages 
for  verification  and  demonstration  of  maintainability  and  integration  of  maintenance 
resources.  The  demonstration  and  validation  should  be  conducted  on  an  integrated 
basis  consistent  with  other  related  test  and  demonstration  requirements.  MIL- 
STD-1388-1  tasks  can  be  substituted  for  MIL-STD-1369. 
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STAGE  ONE 


Stage  one  should  be  progressively  implemented  in  accordance  with  MIT 
STD-471  during  breadboard  or  mock-up,  and  should  continue  through  evaluation  of 
the  first  production  end  article.  For  the  system/equipment,  with  its  subsystems  and 
support  equipment,  the  contractor  will  evaluate  accessibility,  simplicity,  equipment 
size,  working  environment,  maintenance  resource  requirements,  and  human  engineer¬ 
ing  consideration,  and  determine  whether  the  operational  requirements  can  be  met 
without  exceeding  programmed  maintenance  resources. 

Within  30  days  after  the  verification  and  demonstration,  the  contractor  will 
submit  a  report  of  the  verifications  to  the  Administrative  Contracting  Officer.  The 
report  should  include  all  pertinent  data  and  observations,  photographs  or  sketches  of 
mqjor  problem  areas,  and  recommendations  for  corrective  action  as  required.  Subse¬ 
quently,  maintainability  predictions  and  maintenance  resource  requirements  data 
contained  in  the  Logistics  Support  Analysis  (LSA)  should  be  updated. 

STAGE  TWO 

Stage  two  will  occur  concurrently  with  the  test  program  during  which  the 
time  and  achievement  of  the  end  article  maintainability  and  other  logistic  support 
requirements  will  be  verified.  The  verification  will  bo  performed  on  a  test  system  as 
specified  in  the  test  plan.  The  specific  time  phasing,  and  the  maintainability  and 
other  logistic  support  requirements  to  be  verified  and  demonstrated,  will  be  stipu¬ 
lated  by  the  contractor,  agreed  to  by  the  Naval  Electronic  Systems  Command,  and 
included  in  the  maintainability  program  plan.  The  demonstration  will  utilize  the 
numbers  and  skill  levels  of  personnel  and  maintenance  resources  recommended  by  the 
LSA  and  agreed  to  by  the  government.  Publications  and  support  equipment  will  be 
examined  for  adequacy,  compatibility,  and  capability  to  support e  established  main¬ 
tenance  concept.  Maintainability  predictions  and  maintenance  resource  data  require¬ 
ments  should  be  updated  during  this  stage  also. 

STAGE  THREE 


Demonstration  of  the  in-service  end  article  maintainability  characteristics  and 
integration  of  maintenance  resources  will  be  accomplished  by  the  Naval  Electronic 
Systems  Command.  In-service  verification  will  be  accomplished  using  only  those 
tools,  equipment,  data,  training,  personnel,  and  material  resources  which  have  been 
programmed  and  provided.  The  in-service  demonstration  will  include  the  following 
elements: 

1.  Preparation  and  Application.  The  contractor  must  prepare  the  ILS  Verifi¬ 
cation  and  Demonstration  Plan  to  meet  the  criteria  for  demonstrating 
whether  or  not  the  system  or  equipment  support  requirements  have  been 
attained.  This  plan  must  be  agreeable  to  both  the  contractor  and  the  Navy. 
The  demonstration  will  be  conducted  by  the  government  in  a  typical  opera¬ 
tional  environment  with  contractor  participation  as  necessary  to  assure 
mutual  acceptability  of  test  data  and  the  analysis  thereof.  The  plan  must 


provide  for  assessment  of  system  maintainability  characteristics  as  well  as 
support  factors  related  to  item  downtime;  i.e.,  technical  manuals, 
personnel,  tools,  support  equipment,  maintenance  concept,  and  availability 
and  adequacy  of  required  spares  and  repair  parts. 

2.  Management.  The  US  verification  and  demonstration  will  be  managed 
by  the  government.  A  Demonstration  Control  Board  will  be  established 
to  provide  on-site  management  The  board  will  consist  of  three  to  five 
government  members,  one  of  whom  will  be  designated  as  the  Demonstra¬ 
tion  Director,  and  an  equitable  number  of  contractor  personnel  to  be  pro¬ 
vided  at  contractor  option  and  expense.  The  board  will  be  responsible  for 
assuring  that  maintainability,  maintenance,  and  support  data  are  col¬ 
lected  and  documented  in  accordance  with  established  Navy  policy  (or  an 
approved  modification  thereof  as  may  be  necessary);  determining  the  valid¬ 
ity  of  data  reporting;  and  making  initial  determination  as  to  whether  dem¬ 
onstration  objectives  have  been  satisfied  and  contractual  requirements 
have  been  met 

3.  Demonstration  Location  and  Duration.  The  ILS  demonstration  will  prefer¬ 
ably  be  conducted  at  the  naval  activity  which  will  be  required  to  operate 
and  support  the  first  deliverable  production  system.  The  demonstration 
will  commence  approximately  6  months  after  delivery  of  the  system/equip¬ 
ment  to  the  demonstration  site  and  may  continue  for  about  6  months.  This 
will  allow  for  operational  and  maintenance  famliarization  prior  to  the 
demonstration  and  will  provide  for  a  demonstration  of  sufficient  scope  to 
evaluate  maintenance  and  support  requirements  for  a  broad  operational 
spectrum  of  the  system/equipment. 

4.  Demonstration  Test  Team.  The  demonstration  test  team  will  consist  of 
members  of  the  demonstration  activity  and  the  Demonstration  Control 
Board. 

5.  Demonstration  System  Equipment.  The  system/equipment  to  be  used  in 
the  demonstration  will  be  that  regularly  assigned  to  the  activity  in  support 
of  its  assigned  mission.  No  attempt  will  be  made  to  segregate  a  specific 
group  of  systems/equipments  for  demonstration  purposes.  All  assigned  sys¬ 
tems/equipments  will  be  used,  regardless  of  detailed  configuration,  pro¬ 
vided  that  such  systems/equipments  are  production  configured  and 
delivered  to  the  Navy  for  fleet  training  and  operations.  No  specially  config¬ 
ured  test  system/equipment  will  be  used  for  demonstration  purposes. 

6.  Maintenance  and  Support.  All  maintenance  performed  on  the  demonstra¬ 
tion  system/equipment  will  be  accomplished  by  demonstration  site  person¬ 
nel  or  by  personnel  attached  to  the  supporting  intermediate-level 
maintenance  activity. 

Organizational  and  intermediate-level  maintenance  will  be  performed  in 
accordance  with  the  approved/validated  technical  manuals  and  data  and 
the  support  resources  provided  by  the  system/equipment  ILS  program. 
Depot-level  maintenance  will,  however,  be  performed  in  accordance  with 
such  contractual  requirements  as  may  be  applicable  during  the  period  in 
which  the  demonstration  is  performed.  No  organizational  or  intermediate 
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maintenance  will  be  performed  by  contractor  personnel  during  the  demon¬ 
stration  unless  specifically  requested  by  the  demonstration  control  author¬ 
ity.  Similarly,  no  contractor  advice  or  guidance  will  be  given  to  personnel 
performing  maintenance  unless  so  requested  by  the  control  authority. 

7.  Maintenance  Personnel.  The  composition  of  the  demonstration  test  team 

and  the  extent  of  training  of  the  personnel  involved  cannot  be  specified 
except  by  broad  parameters.  It  is  anticipated  that  the  activity  involved  will 
be '  led  with  a  typical  mix  of  maintenance  personnel,  such  mix  to  fol¬ 

low.  •  closely  as  possible  the  maintenance  and  operating  factors  estab¬ 
lished  for  the  system/equipment.  It  is  also  anticipated  that  a  large  portion 
of  the  organizational  and  intermediate-level  maintenance  personnel  will 
have  received  either  factory  or  Navy  training.  Also,  time  will  be  allocated 
for  on-the-job  training,  as  required,  prior  to  commencement  of  the  demon¬ 
stration. 

8.  Demonstration  Support  Material.  Initial  demonstration  site  surveys  for 
verification  of  the  adequacy  of  logistic  support  status  will  be  conducted  by 
the  Demonstration  Control  Board  not  later  than  60  days  prior  to  the 
scheduled  commencement  of  the  demonstration.  Items  to  be  furnished  by 
both  contractor  and  government,  based  on  the  approved  Support  Material 
List  (SML),  will  be  delivered  to  the  test  site  at  least  GO  days  prior  to  com¬ 
mencement  of  the  demonstration.  Thirty  days  prior  to  the  start  of  the  dem¬ 
onstration,  the  Demonstration  Control  Board  will  survey  the  availability 
and  serviceability  of  support  material  and  initiate  action  through  the  pro¬ 
gram  management  office  to  fill  shortages  and  replace  unserviceable  mate¬ 
rial.  The  Demonstration  Control  Board  will  also  make  recommendations 
for  add-on  quantities  of  spares  and  repair  parts.  Basis  for  such  recommen¬ 
dation  should  be  to  reduce  potential  program  delays.  Upon  completion  of 
this  survey  and  all  possible  corrective  actions,  a  report  of  remaining  defi¬ 
ciencies  and  a  recommendation  concerning  program  start/delay  will  be 
furnished  to  the  program  manager,  who  will  decide  the  advisability  of  pro¬ 
gram  start  or  delay. 

9.  Maintenance  Data  Collection.  The  collection  of  accurate  data  and  the 
analysis  thereof  are  prerequisites  for  a  successful  demonstration  program. 
The  data  must  have  a  high  degree  of  accuracy  with  a  broad  base  for  analy¬ 
sis.  The  data  system  utilized  to  collect  the  Dedicated  Maintenance  Man- 
Hours  (DMMH)  requirements  will  be  the  Navy  3M  Data  System.  An 
allowance  of  15%,  or  that  percentage  agreed  to  by  the  government  and  the 
contractor  during  contract  negotiation,  for  PF&S  (personal,  fatigue,  and 
supplementary)  time  will  be  used  as  the  factor  to  convert  all  reported 
DMMH  time  to  actual  DMMH  required  time. 

10.  Analysis/Evaluation  and  Report  Results.  Data  derived  from  the  Demon¬ 
stration  Program  should  be  screened  thoroughly  for  accuracy,  classifica¬ 
tion  of  data,  and  verification  of  mathematic  calculation.  Maintainability 
measurements  should  be  computed  as  specified  in  the  demonstration  plan. 
A  final  demonstration  report  must  be  prepared  by  the  demonstration  direc¬ 
tor  and  submitted  to  the  government  program  manager  within  90  days 
after  completion  of  the  demonstration. 
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11.  Change  Incorporation.  In  the  event  change  is  incorporated  during  the  dem¬ 
onstration  period  that  affects  the  maintainability,  reliability,  or  sup- 
portability  of  the  system/equipment,  the  contractor  may  request  a 
reevaluation  of  the  applicable  demonstration  results  to  that  point  in  time, 
provided  the  change  is  incorporated  and  demonstrated  prior  to  preparation 
and  submission  of  the  demonstration  report. 


RELIABILITY  DEMONSTRATION 


All  test  programs  require  a  test  plan.  On  part  of  the  reliability  test  plan  is  the 
demonstration  of  achieved  reliability.  MIL-STD-785  specifies  the  applicable  portions 
of  MIL-STD-781  for  the  demonstration  test  procedure.  The  reliability  test  plan,  in¬ 
cluding  the  demonstration  test  procedure,  must  be  approved  by  the  procuring  activity 
prior  to  initiation  of  the  tests.  Reliability  tests  should  be  integrated  with  other  test 
programs  to  avoid  costly  duplication  (e.g.,  performance  testing,  flight  testing,  and 
maintainability  demonstration). 

The  test  plan  establishes  ground  rules  for  conducting  tests  (including  GFE  im¬ 
pact),  and  accept/reject  criteria  in  accordance  with  MIL-STD-781.  The  plan  also  in¬ 
cludes  such  items  as  demonstration  milestones,  confidence  or  risk  levels,  tradeoff 
curves,  and  sample  forms. 

The  test  level  may  be  in  accordance  with  MIL-STD-781,  or  an  environmental 
profile,  whichever  is  applicable. 

Along  with  the  reliability  demonstration  (i.e.,  the  actual  test),  there  is 
recorded  data  in  the  form  of  an  operation  log,  failure  reports,  and  failure  analysis 
reports,  plus  a  log  of  equipment  failures  and  operating  time.  These  are  described  in 
MIL-STD-781.  This  failure  data  can  be  essential  because  highly  reliable  items  cannot 
be  tested  efficiently. 


MAINTAINABILITY  DEMONSTRATION 


To  prove  the  achievement  of  the  specified  maintainability  requirement,  a 
maintainability  demonstration  will  normally  be  accomplished  in  accordance  with 
MIL-STD-471.  The  contractor  must  prepare  and  submit  a  demonstration  plan  and 
report  to  the  procuring  activity.  MIL-STD-471  establishes  procedures,  test  methods, 
and  requirements  for  verification,  demonstration,  and  evaluation  of  the  achievement 
of  the  maintainability  requirement.  The  formal  demonstration  is  conducted  in  an  op¬ 
erational  or  simulated  operational  environment  as  specified  in  the  contract.  The  dem¬ 
onstration  should  be  integrated  with  other  testing  requirements  such  as  proof  of 
design,  breadboard,  prototype,  environmental,  production,  and  acceptance.  Data  from 
the  demonstration  will  be  used  to  incrementally  verify  the  achievement  of  maintain¬ 
ability  design  requirements  and  to  update  the  parameter  values  from  the  maintain¬ 
ability  analyses  and  predictions. 
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HUMAN  ENGINEERING  TESTING 


GENERAL 


Tests  are  conducted  to  verify  that  designs  of  the  equipment,  software,  facili¬ 
ties,  and  environment  meet  human  engineering  and  life  support  criteria  and  are  com¬ 
patible  with  the  overall  system  requirements.  The  test  and  evaluation  program  is  to 
be  included  in  the  Human  Engineering  Program  Plan.  Detailed  information  on  test 
and  evaluation  can  be  found  in  MIL-H-46855. 

STUDIES,  EXPERIMENTS,  AND  LABORATORY  TESTS 


The  contractor  shall  conduct  experiments,  laboratory  tests  including  dynamic 
simulation,  and  studies  required  to  resolve  human  engineering  and  life  support  prob¬ 
lems  specific  to  the  system.  Human  engineering  and  life  support  problem  areas  shall 
be  brought  to  the  attention  of  the  procuring  activity  and  shall  include  the  estimated 
effect  on  the  system  if  the  problem  is  not  studied  and  resolved.  These  experiments, 
laboratory  tests,  and  studies  shall  be  accomplished  in  a  timely  manner;  i.e.,  in  a  man¬ 
ner  such  that  the  results  may  be  incorporated  in  equipment  design.  The  performance 
of  any  mqjor  study  effort  shall  require  approval  by  the  procuring  activity. 

Mock-ups  and  Models 

At  the  earliest  practical  point  in  the  development  program,  and  well  before 
fabrication  of  system  prototypes,  full-scale,  three-dimensional  mock-ups  of  equipment 
involving  critical  human  performance  (such  as  an  aircrew  compartment,  maintenance 
work  shelter,  or  command  control  console)  shall  be  constructed.  The  proposed  Human 
Engineering  Program  Plan  shall  specify  mock-ups  requiring  procuring  activity  ap¬ 
proval  and  modification  to  reflect  changes.  The  workmanship  shall  be  no  more  elabo¬ 
rate  than  is  essential  to  determine  the  adequacy  of  size,  shape,  arrangement,  and 
panel  content  of  tho  equipment  for  use  by  man.  The  most  inexpensive  materials  prac¬ 
tical  shall  be  used  for  fabrication.  These  mock-ups  and  models  shall  provide  a  basis 
for  resolving  access,  workspace,  and  related  human  engineering  problems,  and  incor¬ 
porating  these  solutions  into  system  design.  In  those  design  areas  where  equipment 
involves  critical  human  performance  and  where  human  performance  measurements 
are  necessary,  functional  mock-ups  shall  be  provided,  subject  to  prior  approval  by  the 
procuring  activity.  The  mock-ups  shall  be  available  for  inspection  as  determined  by 
the  procuring  activity.  Upon  approval  by  the  procuring  activity,  scale  models  may  be 
substituted  for  mock-ups.  Disposition  of  mock-ups  and  models,  after  they  have  served 
the  purposes  of  the  contract,  shall  be  as  directed  by  the  procuring  activity. 

Dynamic  Simulation 

Dynamic  simulation  techniques  shall  be  utilized  as  a  human  engineering 
design  tool  when  necessary  for  the  detail  design  of  equipment  requiring  critical 
human  performance.  Consideration  shall  be  given  to  use  of  various  models  for  the 
human  operator,  as  well  as  man-in-the-loop  simulation.  While  the  simulation 
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equipment  is  intended  for  use  as  a  design  tool,  its  potential  relationship  to,  or  use  as, 
training  equipment  shall  be  considered  in  any  plan  for  dynamic  simulation. 


HUMAN  ENGINEERING  IN  TEST  AND  EVALUATION 


The  contractor  shall  establish  and  conduct  a  test  and  evaluation  program  to: 
(1)  assure  fulfillment  of  applicable  requirements;  (2)  demonstrate  conformance  of  sys¬ 
tem,  equipment,  and  facility  design  to  human  engineering  design  criteria;  (3)  confirm 
compliance  with  performance  requirements  where  man  is  a  performance  determinant; 
(4)  secure  quantitative  measures  of  system  performance  which  are  a  function  of  man- 
machine  interaction;  and  (5)  determine  whether  undesirable  design  or  procedural  fea¬ 
tures  have  been  introduced.  (The  fact  that  these  functions  may  occur  at  various 
stages  in  system  or  equipment  development  shall  not  preclude  final  human  engineer¬ 
ing  verification  of  the  complete  system.  Both  operator  and  maintenance  tasks  shall  be 
performed  as  described  in  approved  test  plans  during  the  final  system  test.) 

Planning 

Human  engineering  testing  shall  be  incorporated  into  the  test  and  evaluation 
program  and  shall  be  integrated  into'  engineering  design  tests,  contractor  demonstra¬ 
tions,  R&D  acceptance  tests,  and  other  mqjor  development  tests.  Compliance  with 
human  engineering  requirements  shall  be  tested  as  early  as  possible.  Human  engi¬ 
neering  findings  from  early  testing  shall  be  used  in  planning  and  conducting  later 
tests. 

Implementation 

Tho  human  engineering  test  and  evaluation  program,  contained  in  approved 
test  plans,  shall  be  implemented  by  the  contractor.  Test  documentation  (e.g.,  check¬ 
lists,  data  sheets,  questionnaires,  schedules,  operating  procedures,  and  test  proce¬ 
dures)  shall  be  available  at  the  test  site.  Human  engineering  portions  of  all  tests  shall 
include,  where  applicable,  the  following 

A  simulation  (or  actual  conduct  where  possible)  of  mission  or  work  cycle. 

Tests  in  which  human  participation  is  critical  with  respect  to  speed,  accuracy, 
reliability,  or  cost. 

A  representative  sample  of  noncritical  scheduled  and  unscheduled  mainte¬ 
nance  tasks. 

Proposed  job  aids. 

Utilization  of  personnel  who  are  representative  of  tho  range  of  the  intended 
military  user  population  in  terms  of  skills,  size,  and  strength;  and  wearing 
suitable  military  garments  and  equipment  appropriate  to  the  tasks  and 
approved  by  tho  procuring  activity. 

Collection  of  task  performance  data. 
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Identification  of  discrepancies  between  required  and  obtained  task  perform¬ 
ance. 

Criteria  for  the  acceptable  performance  of  the  tests. 

Failure  Analysis 

All  failures  occurring  during,  or  as  a  result  of,  test  and  evaluation  shall  be 
subjected  to  a  human  engineering  review  to  differentiate  between  failures  due  to 
equipment  alone,  to  man-equipment  incompatibilities,  and  to  human  error.  The  pro¬ 
curing  activity  shall  be  notified  of  design  deficiencies  which  contribute  to  human 
error. 


SAFETY  TESTING 


The  system  safety  engineering  activities  are  required  to  establish  test  require¬ 
ments  and  ensure  that  safety  verification  of  design  and  data  are  included  in  the  engi¬ 
neering  test  program. 

The  System  Safety  Program  Plan  (SSPP)  includes  the  manner  of  demonstrat¬ 
ing  quantitative  system  safety  requirements;  identification  of  special  safety  test  data; 
and  range,  flight,  and  operational  test  safety  programs. 

Tests  shall  be  proposed  in  the  SSPP  to  validate  the  safety  of  the  product, 
including  those  tests  already  specified  by  the  procuring  activity.  Safety  tests  shall  be 
integrated  into  appropriate  test  plans.  Where  complete  safety  testing  costs  would  be 
prohibitive,  partial  design  verifit_  'on  of  safety  characteristics  or  procedures  may  be 
demonstrated  by  laboratory  test,  functional  mock-ups,  or  model  simulation,  when 
approved  by  the  procuring  activity.  Safety  tests  shall  be  performed  on  critical  devices 
or  components  to  determine  the  degree  of  hazard  or  margin  of  safety  of  design. 
Induced  or  simulated  failures  will  be  considered  for  demonstrating  the  failure  mode  of 
critical  components.  The  detailed  test  plans  for  all  tests  shall  be  reviewed  to  ensure 
that: 


•  Safety  is  adequately  demonstrated. 

•  The  testing  will  be  carried  out  in  a  safe  manner. 

•  All  additional  hazards  introduced  by  testing  procedures,  instrumentation, 
test  hardware,  etc.,  are  properly  identified  and  minimized. 

A  system  safety  checklist  is  required  as  part  of  the  Contract  Data  Require¬ 
ments  List  (CDRL).  The  checklist  provides  assurance  that  all  required  and  identified 
safety  requirements  have  been  incorporated  in  the  design  and  hardware  and  verified 
by  review,  test,  or  other  method  approved  by  the  agency  concerned. 
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ENVIRONMENTAL  TESTING 


ENVIRONMENTAL  CHARACTERISTICS 


Establishing  minimum  requirements  for  environmental  characteristics  is  espe¬ 
cially  significant  to  low-cost  new-equipment  development.  At  present,  all  equipments 
intended  for  shipboard  use  are  supposed  to  be  subjected  to  the  analysis  and  testing 
rigors  of  MIL-E-16400  for  shock,  vibration,  temperature,  and  humidity.  For  majy 
shipboard  equipment  applications,  however,  MIL-E-16400  requirements  are  unneces¬ 
sarily  rigorous  and  consequently  result  in  unnecessarily  expensive  equipments.  For 
this  reason,  the  TELCAM  study  has  developed  vibration  and  temperature/relative-hu¬ 
midity  tests  less  severe  than  those  required  by  MIL-E-16400  —  tests  which  will  pro¬ 
vide  confidence  in  equipment  endurance  under  usual  shipboard  environments  instead 
of  survivability  under  abnormal,  seldom-occurring  environmental  conditions.  (The 
TELCAM  vibration  tests,  when  passed,  also  serve  to  indicate  acceptable  performance 
under  normal  shock  conditions.)  These  TELCAM  tests  can  be  substituted  for  MIL- 
E-16400  tests  when  the  equipment  under  development  (1)  is  noncritical  or  is  not  vital 
to  survival  of  nor  essential  to  the  mission  of  its  ship,  and  (2)  is  to  be  exposed  to 
shock,  vibration,  temperature,  and  humidity  conditions  found  inside  the  ship  forward 
of  the  propeller  shaft(s).  This  will  help  to  reduce  equipment  development  and  procure¬ 
ment  costs  and  yet  provide  an  equipment  which  will  perform  adequately  under  all 
normal  environmental  conditions.  Appendix  A  discusses  the  TELCAM  approach  to 
environmental  requirements  and  presents  the  test  procedures  to  be  used  when  speci¬ 
fying  the  quality  assurance  provisions  for  required  TELCAM  environmental  charac¬ 
teristics. 

ESTABLISHING  SHOCK  AND  VIBRATION  CHARACTERISTICS 


To  establish  the  minimum  satisfactory  environmental  characteristics  for  shock 
and  vibration,  refer  to  figure  V-l  and  the  following  options. 

Option  I.  For  noncritical  equipments  to  be  used  only  in  Area  III  —  the  least 
vibrationally  severe  area  —  of  the  ship(s)  on  which  the  equipments  are  to  be  in¬ 
stalled,  establish  vibration  requirements  which  the  equipment  must  meet  after  being 
subjected  to  the  test  presented  in  appendix  A,  the  numerical  values  of  which  are  given 
in  table  A-I  of  the  appendix.  The  test  is  a  broad  one,  covering  the  Area  III  vibration 
characteristics  of  all  ships.  On  some  ships,  however,  these  characteristics  are  less 
severe  than  on  others.  When  the  equipment  is  to  be  used  exclusively  in  a  ship  or 
ships  having  less  severe  vibration  characteristics  than  those  for  which  the  test  is 
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intended,  the  test  (and  requirements)  can  be  made  less  constraining  by  substituting 
the  specific  ship  characteristics,  presented  in  figures  A-l  through  A-8  of  appendix  A, 
in  place  of  the  values  in  table  A-l. 


Figure  V-t.  Shipboard  environmental  areas. 


Option  2.  For  noncritical  equipments  to  be  used  primarily  in  Area  III  but  with 
some  to  be  located  in  Area  I  or  II,  establish  the  option  1  vibration  test  requirements, 
and,  in  addition,  for  the  percentage  of  equipments  in  Areas  I  and  II,  establish  sepa¬ 
rate  packaging  and  testing  requirements  that  will  allow  these  equipments  to  meet 
MIL-STD-167  (see  fig  A-l  in  appendix  A).  When  the  costs  for  extra  packaging  for  the 
required  number  of  equipments  equals  or  exceeds  80%  of  the  cost  that  would  be  re¬ 
quired  to  extra-package  all  the  equipments,  then  establish  shock  and  vibration  re¬ 
quirements  for  all  equipments  as  per  the  following  option  3. 

Option  3.  For  noncritical  equipments  to  be  used  entirely  or  almost  entirely  in 
Area  I  or  II,  and  for  all  critical  equipments,  establish  the  level  and  testing  require¬ 
ments  of  MIL-STD-167  for  vibration  and  of  MIL-S-901  for  shock.  Instead  of  making 
general  reference  to  these  documents,  pinpoint  only  those  requirements  necessaiy  for 
survival  by  citing  specific  paragraphs. 
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ESTABLISHING  TEMPERATURE  AND  HUMIDITY 
CHARACTERISTICS 


To  establish  the  minimum  satisfactory  environmental  characteristics  for  tem¬ 
perature  and  humidity,  refer  to  figure  V-2  and  the  following: 

1.  For  noncritical  equipments  to  be  located  inside  a  ship  (Areas  I  and  III,  fig 
V-l)  —  i.e.,  for  those  class  4  (MIL-E-16400)  equipments  which  are  not 
exposed  directly  to  weather  —  establish  the  temperature-humidity  require¬ 
ments  profiled  by  figure  V-2  and  test  by  malting  five  complete  cycles 
around  the  trapezoid  of  the  figure.  Each  test  cycle  starts  and  finishes  at 
95°F  and  95%  relative  humidity,  with  each  of  the  test  condition  points 
being  maintained  for  5  hours  and  with  1  hour  allowed  for  transition 
between  the  points  —  a  24-hour  period. 

2.  For  equipments  to  be  located  in  more  exposed  areas  —  i.e.,  for  class  1,  2, 
and  3  (MIL-E-16400)  equipments  —  establish  the  temperature  and  humid¬ 
ity  requirements  imposed  by  MIL-E-16400  for  the  corresponding  equip¬ 
ment  class  (refer  to  paragraphs  3.13  and  4.6.8  of  MIL-E-16400H). 
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Figure  V-2.  TELCAM  temperature-humidity  test. 


ESTABLISHING  OTHER  ENVIRONMENTAL  CHARACTERISTICS 

Other  environmental  constraints  concern  noiso,  enclosures,  inclination,  wind 
speed,  EMI,  etc.,  as  covered  by  MIL-E-16400.  They  may  or  i.  .y  not  have  to  be  char¬ 
acterized  and  tested,  depending  on  the  operational  requirements  of  the  equipment 
characteristics  being  established.  If  they  do,  the  requirements  of  MIL-E-16400  should 
be  cited.  Refer  to  figure  V-3. 
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Flguf  •  V-3.  Environmental  documents. 
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To  be  especially  noted  is  that  airborne  and  structurebome  noise  and  enclosure 
requirements  and  tests  should  be  applied  to  all  sheltered  equipments.  Inclination  re¬ 
quirements  and  tests  should  be  applied  only  to  equipments  whose  proper  operation 
depends  on  fluid  levels,  position-sensitive  switches,  or  heat  pipes,  or  which  arc  other¬ 
wise  position  sensitive,  since  this  constraint/test  has  a  motion  to  which  most  equip¬ 
ments  are  insensitive. 

ENVIRONMENTAL  REFERENCES 

1.  Naval  Air  Systems  Command 
Military  Standard  MIL-STD-810C, 

Environmental  Test  Methods,  10  March  1975 

2.  Naval  Ship  Engineering  Center 
Military  Standard  MIL-STD-1399A, 

Interface  Standard  for  Shipboard  Systems,  20  Dec  72 

3.  Naval  Electronic  Systems  Command 

Military  Specification  MIL-E-2036C,  Enclosure  for 

Electric  and  Electronic  Equipment,  Naval  Shipboard,  15  Mar  63 

4.  Naval  Ordnance  Systems  Command 
Military  Specification  MIL-C-456G2A, 

Calibration  System  Requirements,  9  Feb  62 

5.  Naval  Ship  Engineering  Center 

Military  Standard  MIL-STD-740B,  Airborne  and 
Structurebome  Noise  Measurements  and  Acceptance 
Criteria  of  Shipboard  Equipment,  13  Jan  65 

6.  Naval  Ship  Engineering  Center 

Military  Standard  MIL-STD-108E,  Definitions  of  and 
Basic  Requirements  for  Enclosures  for  Electric  and 
Electronic  Equipment,  4  Aug  66 

7.  Naval  Air  Systems  Command 

Military  Specification  MIL-E-6051D,  Electromagnetic 
Compatibility  Requirements,  Systems,  7  Sep  67 

8.  Air  Force  Logistics  Command 

Military  Specification  MIL-R-9673B,  Radiation  Limits, 

Microwave  and  X-Radiation  Generated  by  Ground  Electronic 
Equipment  (as  Related  to  Personnel  Safety),  15  Sep  61 

9.  Naval  Ordnance  Systems  Command 

Military  Standard  MIL-STD-1385,  Preclusion  of  Ordnance 
Hazards  in  Electromagnetic  Fields,  General  Requirements  for,  6  Apr  72 

10.  Naval  Air  Systems  Command 

Military  Standard  MIL-STD-704A,  Electric  Power, 

Aircraft,  Characteristics  and  Utilization  of,  9  Aug  66 
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11.  Naval  Ship  Engineering  Center 

Military  Standard  MIL-STD463,  Definitions  and  Systems  of 
Units,  Electromagnetic  Interference  Technology,  9  Jun  66 

12.  Naval  Facilities  Engineering  Command 
Military  Standard  MIL-STD-633C, 

Mobile  Electric  Power  Engine  Generator  Standard 
Family  Characteristics  Data  Sheet,  29  Oct  71 

13.  Naval  Air  Systems  Command 
Military  Standard  MIL-STD-831,. 

Test  Reports,  Preparation  of,  28  Aug  63 

14.  Naval  Electronic  Systems  Command  Military 
Standard  MILrSTD461A,  Electromagnetic  Interference 
Characteristics,  Requirements  for  Equipment,  1  Aug  68 

15.  Naval  Electronic  Systems  Command  Military  Standard 
MIL-STD462,  Electromagnetic  Interference 
Characteristics,  Measurement  of,  31  Jul  67 

16.  Naval  Ship  Engineering  Center  Military  Standard 
MIL-STD469,  Radar  Engineering  Design  Requirements, 
Electromagnetic  Compatibility,  1  Dec  66 

•  17.  Naval  Ship  Engineering  Center  Military 
Specification  MIL-S-901C,  Shock ‘Tests,  H.I. 

(Iligh-Impact),  Shipboard  Machinery,  Equipment 
and  Systems;  Requirements  for,  5  Sep  63 

18.  Naval  Ship  Engineering  Center  Military  Standard 
MIL-STD-167/1,  Mechanical  Vibration  of 
Shipboard  Equipment,  1  May  74 

19.  Vibration  Test  Data  Reports: 


Source 

Tills 

Report  No 

Date 

Boston  Naval 
Shipyard: 

1.  “Collection  of 

1961 

Reports  on  Vibra¬ 
tion  Surveys* 

2.  same 

1962 

3.  same 

— 

1963 

4.  same 

AD  458905 

1964 

5.  same 

AD  479764 

1965 

6.  same 

AD  813701 

1966 

7.  same 

— 

1969 
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Source 

Title 

BmiUIa 

Date 

Long  Beach  Naval 

Shipyard: 

1.  “Collection  of 
Reports  of  Vibra¬ 
tion  Surveys" 

— 

1961 

2.  “Collection  of 
Vibration 

— 

1962 

Reports" 

3.  ‘Collection  of 
Reports  of  Vibra¬ 
tion  Surveys 

- 

1963 

4.  same 

AD460569 

1964 

5.  same 

AD  428086 

1965 

6.  same 

AD  827529 

1967 

7.  same 

AD  847891 

1968 

8.  same 

— 

1969 

9.  same 

_ 

1970 

10.  “USS  Iwo 

Jima  (LPH  2) 
Underway 
Vibration  Sur¬ 
vey" 

1972 

Norfolk  Naval 

Shipyard: 

1.  “Vibration 
Survey  Report 

1961 

2.  same 

— 

1962 

3.  same 

— 

1963 

4.  same 

— 

1964 

5.  same 

AD  482035 

1965 

6.  same 

AD  818880 

1966 

7.  same 

AD  834603 

1967 

8.  same 

— 

1968-1969 

9.  same 

— 

1972 

Pearl  Harbor 

Naval  Shipyard: 

1.  “Vibration 
and  Noise  Sur¬ 
vey  Reports* 

— 

1961 

2.  same 

— 

1963 

Philadelphia  Naval 

Shipyard: 

1.  “Collection  of 
Letter  Reports  of 
Local  Vibration 
Surveys’ 

1961 

2.  same 

— 

1962 

3.  same 

_ 

1963 

4.  “Collection  of 
Vibration  Sur¬ 
veys’ 

AD  809229 

1964 
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Source 


Report  No 


litis 


Eats 


5.  “Collection  of 
Vibration  Sur¬ 
veys  and  Letter 

Pnnftrfc* 


6.  “Collection  of 
Vibration  Sur¬ 
veys’ 


Portsmouth  Naval 
Shipyard: 

1.  “Reports  on 
Vibration  Sur¬ 
veys’ 

Puget  Sound 
Naval  Shipyard: 

1.  “Collection  of 
Reports  on 
Vibration  Sur¬ 
veys" 

2-  same 
3.  same 
4:  samo 

5.  same 

6.  same 

7.  same 

San  Francisco  Bay 
Naval  Shipyard: 

1.  “Shipboard  Vibra-  — 

tion  Survey  of 

1962” 

2.  "Collection  of  — 

Vibration  Sur¬ 
veys  for  1963’ 

3.  “Shipboard  and  AD  466652 
Vibration 

Memos  and  Sur¬ 
veys  for  1964” 

4.  “Collection  of  — 

Vibration  Sur¬ 
veys  for  1965’ 

5.  same...  “1966’  AD  815849 

Charleston 
Naval  Shipyard: 

1.  “Informal  and  AD  460923 

Letter  Reports 

on  Vibration 

Problems  for 

1964’ 


AD  463274 
AD  481976 
AD  809213 
AD  834332 


1969 

1970 


1962 


1961 


1962 

1963 

1964 

1965 

1966 

1967 


1962 

1963 

1964 

1965 

1966 


1964 
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Source  Title  Report  No  Date 

Naval  Electronics 
Laboratory  Center; 

1.  D.  Peterson,  1701  2  Apr  1970 

‘Summary  of 

NELC’, 

Recorded  Ship¬ 
board  Vibra¬ 
tion  Data 

2.  RJL  Chalmers  1577  16  Aug  1968 

and  D.L.  Peterson 

“Environmental 
Studies  Aboard 
U.S.  Navy  Ves¬ 
sels  in  the  South 
China  and  Carib¬ 
bean  Seas” 

Naval  Ship  Research 
and  Development  Center, 

Washington,  D.C. 

1. V.S.  Hardy,  NSRDC  2338  June  1967 

“Surface  Ship 

Vibration” 

2.  H.b .  Alma  and  NSRDC  2338A  Sep  1970 
N.W.  Huzil,  “Surface 

Ship  Vibration’ 

29.  NELC  Memo  Z269,  Ser  4700-M205-73  dated  27  December  1973,  from  TA 
Danielson  to  DA  Peterson  (Subj:  Supply  Line  Voltage  and  Frequency  Test 
Results  on  Century  Data  Co.  “Floppy  Disk  Memory  Unit”  Ser  No  512) 

'21.  NELC  Letter  Z269,  Ser  4700-82  date  5  December  1973,  from  Commander 
NELC  to  Commander  NUC  (Subj;  Results  of  exploratory  vibration  test  of 
CALCCMP  Model  565  Plotter) 


INTERFACE  REQUIREMENTS  VALIDATION 


The  need  for  interface  standards  for  shipboard  systems  has  become  apparent 
as  ships  and  their  systems/equipment  have  grown  more  complex  and  as  discrete 
activities  (SYSCOMs,  PMs,  Contractors,  etc.)  involved  in  ship/equipment  design  have 
proliferated.  A  Shipboard  Interface  Standards  Program  has  been  established  to  meet 
this  need.  Shipboard  interfaces,  when  considered  in  their  totality,  confront  the  sys- 
tems/dosign  engineer  with  complex  and  intricate  problems.  Solution  of  these  prob¬ 
lems  is  facilitated  by  defining  particular  selected  interfaces  and  the  constraints  they 
impose  on  equipments.  MIL-STD-1399  is  structured  to  provide  these  definitions.  This 
standard,  and  its  supporting  sections,  defines  the  constraints  on  systems/equipment 


design  imposed  by  the  established  characteristics  of  the  shipboard  environment.  It 
will: 


•  Specify  constraints  for  systems/equipment  design  imposed  by  the  ship- 
boaid  environment 

•  Provide  for  early  dialog  among  personnel  concerned 

•  Assist  in  achieving  mere  effective  ship  configuration  management 

•  Contribute  to  cost-effective  and  integrated  ships  by  promoting  interface 
compatibilify 

•  Ultimately  achieve  interface  compatibility  of  installed  systems/equipment 
with  the  shipboard  environment 

MHrSTD-1399  applies  to  all  activities  involved  in  ship/systems/equipment 
design,  production,  and  installation  and  requires  that  the  interfacing  aspects  of  ship / 
systems/equipment  be  given  priority  consideration  by  such  activities.  The  specific  in¬ 
terface  characteristics  and  constraints  established  in  the  various  sections  of  this 
standard  are  mandatory  and  shall  be  adhered  to  by  SYSCOMs,  PMs,  contractors,  and 
all  others  engaged  in  any  aspect  of  total  shipboard  design,  including  system/equip¬ 
ment  design,  production,  and  installation.  It  is  incumbent  upon  interested  activities, 
in  consonance  with  the  objectives  of  the  Interface  Standards  Program,  to  establish  a 
dialog  in  a  timely  manner  which  will  bring  into  focus  and  resolve  any  interface  prob¬ 
lems  which  may  require  attention  in  areas  not  yet  covered  by  this  standard.  It  is 
essential  that  interface  requirements  be  carefully  considered  by  all  naval  activities 
and  contractors  involved  in  ship  construction/modernization/conversion  throughout 
the  entire  ship  life  cycle.  It  is  also  mandatory  that  Principal  Development  Activities 
(PDAs)  invoke  this  standard  in  the  Development  Plans  (DPs)  and  procurement  speci¬ 
fications  for  new  systems/equipment  destined  for  shipboard  installation. 

REFERENCE 

Naval  Ship  Engineering  Center  Military  Standard,  MIL-STD-1399B,  Interface 
Standard  for  Shipboard  Systems,  22  Nov  77. 


DEFICIENCY  ANALYSIS 


During  the  validation  testing,  analyze  any  failures  or  deficiencies  as  to  their 
impact  on  the  program.  Quick  fixe3  may  have  been  incorporated  during  testing,  and 
further  modification  may  be  required  for  service  use.  The  extent  of  the  deficiencies, 
failures,  or  required  modifications  will  determine  the  course  of  action  at  this  point. 
The  next  step  can  range  from  obtaining  service  approval  to  requesting  more  funds  for 
modification  or  further  development,  or  even  to  canceling  the  program. 

In  the  event  that  modification  is  called  for,  the  program  plan  will  (may)  also 
require  modification.  This  includes  new  schedules,  readjusting  fund  allocations  (new 
or  existing  funds),  and  most  of  the  other  steps  covered  in  chapter  III,  Program  Plan¬ 
ning. 
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The  technical  approach  may  also  need  modification.  Refer  to  chapter  IV,  Con¬ 
ceptual  Phase.  This  will  ultimately  lead  to  updating  the  system  specification.  Major 
changes  in  the  technical  approach  will  require  new  validation  tests.  Refer  to  the  in¬ 
troduction  to  this  section.  Those  elements  of  the  system  that  do  not  change  may  still 
be  covered  by  the  original  validation,  depending  on  the  impact  of  the  change. 
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TRANSITION  TO 
PRODUCTION 


VL  TRANSITION  TO  PRODUCTION 


The  transition  to  production  phase  is  simply  the  translation  of  a  viable 
system  concept  into  a  form  which  can  be  efficiently  produced  in  the 
quantities  required  for  service  use.  Off-the-shelf  systems  or  readily 
modified  systems  involve  a  straightforward  transition;  developments 
may  cause  a  very  complex  transition.  This  phase  is  almost  entirely  dic¬ 
tated  by  the  decisions  made  in  prior  phases;  therefore,  planning  incor¬ 
porating  the  considerations  discussed  in  this  chapter  must  be 
accomplished  much  earlier  in  the  project  cycle,  usually  in  the  concep¬ 
tual  phase. 

Once  the  technical  approach  has  been  validated,  the  program  is  ready  to  com¬ 
plete  the  acquisition  cycle.  At  this  transition  point,  the  decisions  to  build,  buy,  or 
modify  eo,,:pments  to  suit  the  requirements  must  be  finalized. 


Alternative 

Phase 

Mqior  Phase 

Build 

Full-Scale  Development 
(EDM) 

Engineering  Qualification 

Product  Development 

Modify 

Modifications 

Buy 

TECHEVAL/OPEVAL 

Service  Approval 

Production 

Qualification 

Deployment 

Initiation 

While  the  actions  required  in  each  phase  may  differ  among  the  alternatives,  it  is  pos¬ 
sible  to  assemble  a  system  with  equipments  acquired  through  each  alternative;  in 
such  a  case,  system  integration  is  always  a  major  issue  in  the  qualification  phase. 

There  are  two  mqjor  milestones  in  the  transition  to  production  —  Approval 
for  Full  Production  (AFP)  and  Initial  Operational  Capability  (IOC).  Refer  to  chapter 
VII,  Approval  for  Production,  for  the  requirements  for  meeting  the  AFP  milestone. 
IOC  is  the  date  that  the  deployment  phase  is  initiated.  Another  milestone,  Final 
Operational  Capability  (FOC),  delineates  the  completion  date  of  the  deployment 
phase.  IOC  is  extremely  important  because  the  equipment  logistics  support  must  be 
initiated  by  IOC.  In  the  many  instances  in  which  the  support  has  not  been  initiated 
in  consonance  with  the  equipment,  the  result  has  been  a  degrading  of  the  equipment, 
frequently  leading  to  permanently  unacceptable  performance.  IOC  may  be  a  firm  or  a 
double  milestone.  It  is  a  firm  milestone  when  the  equipment  must  be  available  on  a 
specific  date  such  as  when  it  is  GFE  to  a  larger  contractual  effort  (such  as  new-ship 
construction).  Flexible  IOC  exists  when  the  equipment  delivery  is  required  only  to 
meet  mission  deeds.  The  firm  milestone  may  be  slipped  in  actuality;  however,  the 
IOC  must  be  assumed  to  be  fixed  for  planning  purposes.  The  dexible  milestone  may 
actually  be  quite  indexible  because  of  ship  availability  schedules  for  installation  and 
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for  political  reasons.  The  IOC  should  be  established  at  the  initiation  of  the  program 
and  guide  the  program  planning  actions  accordingly;  therefore,  the  flexibility  of  the 
IOC  must  be  known  in  order  to  assess  schedule  risks. 


MAJOR  DECISIONS 


Whichever  acquisition  alternative  is  pursued,  there  are  five  mqjor  issues 
which  must  be  resolved  within  the  program  objectives  and  constraints.  These  are; 

•  System  partitioning/integration 

•  Design  ownership 

•  Specification  type 

•  Level  of  repair  performed  at  the  field  level 

•  Standardization 

Each  of  these  issues  may  be  politically  volatile  depending  upon  the  sponsoring  activ¬ 
ity  and  the  time  within  the  acquisition  cycle  when  a  decision  is  made.  In  general,  the 
earlier  within  a  program  that  each  issue  is  addressed,  the  better  the  prospects  for  a 
smooth  transition.  This  is  true  not  only  from  the  standpoint  of  being  able  to  preplan, 
but  from  the  standpoint  of  political  risk  as  well.  An  early  decision  point  allows  time 
to  budget  properly  and  to  bring  the  parties  involved  in  the  acquisition  to  an  under¬ 
standing  of  the  acquisition  plans;  both  elements  serve  to  mollify  opposing  views. 


SYSTEM  PARTITIONING/INTEGRATION 


The  first  and  most  complex  issue  involves  the  system  partitioning  and  the  sub¬ 
sequent  integration  of  the  pieces  into  the  system.  The  degree  of  partitioning  possible 
for  a  system  is  directly  proportional  to  the  complexity  of  the  system;  a  veiy  complex 
system  will  be  partitioned  within  a  hierarchy  of  levels  of  varying  degrees  of  complex¬ 
ity  as  illustrated  below; 


Lml  Definition 

1  Piece  part 

2  Module 

3  Subassembly  or  Shop  Replaceable  Assembly 
(SRA) 

4  Assembly 

5  Unit  or  Weapon  Replaceable  Assembly  fWRA) 
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Level  Definition 

6  Group 

7  Set 

8  Subsystem 

9  System 


The  definitions  above  conform  to  MIL-STD-280  except  for  Module,  WRA,  and  SRA. 
Module  was  added  in  recognition  of  the  great  increases  in  system  complexity  within 
recent  years;  additional  levels  could  be  accommodated  in  a  like  manner.  WRA  and 
SRA  are  defined  in  MIL-STD-1390.  Obviously,  less-complex  equipments  have  fewer 
levels  of  complexity.  A  sing) r- function  simple  equipment  might  have  only  two  levels 
—  equipment  and  piece  pen.  See  figure  VI-1.  The  necessity  of  breaking  down  the  sys¬ 
tem  for  cost  estimating,  scheduling,  work  package  formulation,  etc.,  is  evident  in  the 
utility  of  the  Work  BrcaJidown  Structure  (WBS)  (see  chapter  III,  Program  Planning). 
Partitioning  is  also  an  important  step  in  resolving  the  other  four  major  issues.  The 
partitioning  phase  normally  takes  place  during  concept  formulation  (see  chapter  IV, 
Conceptual  Phase). 

Once  partitions  are  established,  the  system  integration  problem  begins.  The 
primary  issue  to  be  resolved  regarding  system  integration  is  where  the  responsibility 
for  the  tasks  lies  —  which  commands/activities  within  the  government  and  which 
industrial  companies  have  mqjor  and  subordinate  roles.  The  Systems  Engineer, 
responsible  as  the  system  integrator,  must  do  the  following: 

1.  Transform  an  operational  need  into  description  of  system  performance 
parameters  and  a  system  configuration  through  the  use  of  an  iterative 
process  of  definition,  synthesis,  analysis,  design,  test,  and  evaluation. 

2.  Integrate  related  technical  parameter  and  assure  compatibility  of  all  physi¬ 
cal,  functional,  and  program  interfaces  in  a  manner  which  optimizes  the 
total  system  definition  and  design, 

3.  Integrate  reliability,  maintainability,  safety,  human,  and  other  such  fac¬ 
tors  into  the  total  engineering  effort. 

System  engineering  effort  includes,  for  example,  system  definitization,  overall  system 
design,  design  integrity  analysis,  system  optimization,  cost/effectiveness  analysis, 
weight  and  balance  analysis,  and  intrasystem  and  intersystem  compatibility  analysis. 
It  also  includes  reliability,  maintainability,  safety  and  survivability  program  require¬ 
ments,  human  engineering  and  manpower  factors  program,  preparation  of  equipment 
and  component  performance  specifications,  security  requirements,  logistics  support 
integration,  and  design  of  test  and  demonstration  plans. 

If  the  operational  needs  are  to  be  met  by  the  system  design,  it  is  virtually  im¬ 
possible  for  the  government  to  delegate  these  ultimate  system  eng'neering  responsi¬ 
bilities  to  industiy  since  a  detailed  knowledge  of  the  operational  need  is  required; 
however,  these  responsibilities  normally  are  delegated  to  industry  for  portions  of  the 
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system  below  a  given  level  of  complexity.  Attempts  to  contract  the  ultimate  system 
engineering  tasks  will  nullify  the  government’s  control  of  the  technical  effort  result¬ 
ing  in  a  high  degree  of  risk  than  obtaining  a  product  acceptable  to  the  user.  Even 
when  the  operational  need  can  be  well  described  to  the  contractor,  the  government 
will  not  be  in  a  position  to  determine  optimal  tradeoffs.  When  the  need  is  not  well 
defined,  only  the  government  is  in  a  position  to  estimate  the  elements  needed  to  fill 
in  the  holes.  System  integration  task  responsibilities  should  be  delegated  and  divided 
along  the  same  partition  boundaries  as  the  system  engineering  tasks.  The  division  of 
responsibilities  should  be  clearly  established,  consistent  with  the  determinations 
made  for  the  other  mqjor  issued  and  always  implying  total  industry  accountability  to 
the  government  for  the  assigned  tasks  (i.e.,  the  government  should  not  assume 
responsibility  for  an  item  which  is  to  be  integrated  into  an  item  for  which  industry  is 
responsible  unless  the  partition  interfaces  are  fully  specified  and  validated).  This  lat¬ 
ter  point  is  the  most  controversial,  risky,  and  difficult-to-implement  aspect  of  system 
integration,  and  it  is  the  most  important.  Figure  VI-2  illustrates  a  reasonable  resolu¬ 
tion  of  this  issue. 

One  of  the  mqjor  decisions  to  be  made  is  whether  to  task  the  physical  assem¬ 
bly  of  the  final  system  to  industry  or  to  use  in-house  resources.  If  a  single  contractor 
or  prime  contractor  is  being  utilized,  that  contractor  should  be  tasked;  however,  if 
the  system  consists  largely  of  existing  equipments  from  diverse  sources,  in-house  re¬ 
sources  should  be  utilized.  In  cither  case,  the  government  must  develop  the  documen¬ 
tation  within  its  area  of  direct  integration  responsibilities. 


DESIGN  OWNERSHIP 


The  issue  of  design  ownership  is  extremely  controversial  and  politically  sensi¬ 
tive,  and  therefore  must  be  resolved  with  a  well  documented  and  well  justified  deci¬ 
sion.  The  decision  to  be  made  is  whether  the  system  design  should  be  government 
owned  or  contractor  owned;  the  controversy  arises  because  it  generally  assumed  that 
design  ownership  must  bo  f .  her  one  or  the  other.  Actually,  the  requirements  for  the 
government  to  own  the  design  seldom  dictate  an  exclusive  decision,  and  practical  so¬ 
lutions  can  be  formulated  whereby  the  government  owns  part  of  the  design  and  the 
contractor  owns  the  rest.  The  government  ownership  requirements  are  for  designs 
meeting  one  or  more  of  the  following  conditions: 

1.  The  design  is  based  upon  in-house  expertise  which  far  exceeds  industrial 
expertise. 

2.  Design  features  closely  interact  with  service  doctrine  and  directly  affect 
the  user’s  performance  of  duty. 

3.  Configuration  control  must  be  maintained  for  repair  and  standardization 
purposes. 

4.  There  is  essentially  no  commercial  market,  and  government  requirements 
are  recurring  but  insufficient  to  support  continuous  production  or  multi¬ 
source  production. 
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LEVEL  OP  COMPLEXITY 


SYST£M  PARTTTXJMNO 
wmf  MAJOR  ISSUES  RESOLVED 
SYSTEM 


8  SUBSYSTEM  1 

GOVERNMENT  RESPQNSiStiTY 

7  MXISTRY  RESPONSI84JTY  SET  11(2)  SET  12  (X)  SET  13(X) 

6 

5  WTT  1101  (X)  ^UNTYiazT’T*  UNIT  1103  (X)  UMT  1201  ONTT  1301 

4  ASSY  11011-ASSY  11014  ASSY  11021  ASSY  11031  .STD  ASSYS  _  STD  ASSYS 

3  S/ASSY  110211* S/ASSY  110216 

2  _  sfo  mocu.es"'””  "std  modules’ 


suesv 


GROUP  201  (0)  GROUP  211 

UNH2I11 


ASSY  20111  ASSY  20112  ASSY2111l(»)  ASSY21112(X 


STO, ASSYS  STO  ASSYS 


MOOULES  .  „SJD  MOOOES  _  STD  MODULES 


1  PIECE  PARTS. 


PLE  SYSTEM  PARTTOOMNG 
<  MAJOR  ISSUES  RESOLVEO 
TEM 


ueSYSTI 

SUBSYSTEM  2 

SUBSYSTEM  3 

SET  21 

SET  31  SET  32  (0) 

GROUP  211 

GROUP  212  (X) 

^  _ 

UMT  2111 

UNIT  2121  (*) 

UNIT  2122 

UNTT310NX)  UNTT320 

12  (X)  ALSSY  21111  (♦)  ASSY  21112  (X)  ASSY 21211  ASSY  21212  ASSY21221  ASSY  2122  ASSY 310!  1-31018  ASSY 3201 


S/ASSY  212221  S/ASSY  212222 


ES  ~Q  MCXXXES^  STD  MOOSES  STD  MCXXA.ES^  MOa  21 222J  1*2122217  STO  MODULES 


- a 

(3)r  ~  — 


NOT  NOTES’  (1)  THE  LEVEL  OF  DESIGN  OWNEPSXP  OETAX.S 

EVERY  DESIGN  CHARACTERISTIC  ABOVE  THE 
LEVEL  AMO  FUNCTIONALLY  DESCRIBES  THE  ITEMS 
BELOW. 


(2)  THE  TWO  COMMERCIAL  UNITS  ARE  PROCUREO 
BY  A  CONTRACTOR  ANO  INTEGRATED  NTO  AN 
ITEM  WTTHN  THE  CONTRACTOR  S  TASKED 
RESPONSIBILITY,  F  UNIT  1102  WERE  SUPPLIED 
GTE,  THE  GOVERNMENT  WOULD  ASSUME  INTE¬ 
GRATION  RESPONSSUTES  FOR  SET  11. 

(3)  THS  PORTION  OF  THE  SYSTEM  NCLUOES  A 
NUMBER  OF  SYSTEM  PECULIAR  FUNCTIONS  WHCH 
REOORE  CLOSE  CONFIGURATION  CONTROL. 

(4)  THE  LEVEL  OF  STANDARDIZATION  IS  SET  TO 
CONFORM  TO  i VC  LOWEST  OF  LOR  OR  LOOO 
EACH  PARTITION  ABOVE  THS  LEVEL  IS  CON¬ 
TROLLED  BY  A  FUNCTIONAL  SPECULATION  FOR 
STANDARDIZATION  PURPOSES. 


GOVERNMENT  /INDUSTRY  SYSTEM 
INTEGRATION  BOUNDARY 

LEVEL  OF  DESIGN  OWNERSHIP  (1) 

.  .  .  . 

LEVEL  OF  REPAIR 

LEVEL  OF  STANDARDIZATION  (4) 

n 

COMMERCIAL  EQUIPMENT 

(X) 

INDUSTRY  DEVELOPED 

<♦> 

EXISTING  MILITARY  EQUIPMENT 

(0) 

IN-ROUSE  DEVELOPED 

Figure  Vl-2.  Sample  system  partitioning  with  major  Issues  resolved. 
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The  above  conditions  dictate  that  the  government  should  own  at  least  a  por¬ 
tion  of  all  designs.  However,  condition  (2)  features  can  often  be  specified  in  a  func¬ 
tional  specification  without  regard  to  the  design  implementing  those  features,  and 
condition  (3)  applies  only  to  the  level  of  standardization  which  must  be  maintained. 
Therefore,  under  many  conditions  the  government  is  only  interested  in  owning  a  por¬ 
tion  of  the  design  —  usually  the  functional  design  down  to  some  level  of  complexity 
supplemented  by  the  detailed  (fabrication)  design  in  limited  portions  of  the  system 
meeting  one  of  the  above  conditions  (usually  (1)  or  (3)). 

Detailed  design  ownership  is  always  expensive  to  acquire.  Furthermore, 
detailed  designs  dictate  significantly  more  government  responsibility  in  the  develop¬ 
ment,  qualification,  and  use  of  the  design.  Specifically,  the  government  must  procure 
more  documentation  and  assume  the  responsibility  that  the  documentation  is  accu¬ 
rate  when  it  is  utilized  in  production.  Also,  proprietary  parts  and  processes  which  are 
common  to  sophisticated  technologies  must  be  excluded  from  the  design  or  else  the 
government  must  obtain  license  and  documentation  for  the  part  or  process.  This 
negates  the  advantages  of  the  proprietary  feature  for  the  contractor  and  therefore  is 
hard  to  obtain.  The  other  option  for  the  government  is  to  be  satisfied  with  a  perma¬ 
nent  sole-source  situation  which  will  probably  negate  any  advantage  to  design  owner¬ 
ship.  More  aspects  of  detailed  design  ownership  are  discussed  as  part  of  the  tradeoff 
between  functional  and  fabrication  specifications  (see  SPECIFICATION  TYPE, 
below).  In  general,  detailed  design  ownership  requires  close  monitoring  of  tho  design 
development  by  technically  knowledgeable  government  personnel,  validation  of  tho 
developed  design  to  prove  conformance  to  the  functional  design  requirements  and  to 
demonstrate  the  reproducibility  of  the  design,  maintenance  of  an  up-to-date  configu¬ 
ration,  and  exclusion  of  proprietary  parts  and  processes.  The  validation  of  the  design 
is  extremely  important  and  consists  of  an  unbiased  analysis  of  the  allowable  toler¬ 
ances  of  each  part  to  ensure  conformance  to  tho  functional  design  under  worst-case 
conditions.  This  may  consist  of  a  “simple’  study  for  straightforward  designs.  How¬ 
ever,  complex  designs  or  designs  utilizing  complex  processes  may  require  the  building 
of  hardware  by  a  government  facility  or  by  another  contractor  to  proof  the  design 
data  package. 

As  the  complexity  of  a  design  increases,  the  risks  associated  with  not  validat¬ 
ing  the  design  data  package  increase  rapidly.  These  risks  manifest  themselves  as  a 
probability  that  the  design  will  not  perform  the  required  functions  or  that  tho  design 
will  fail  to  fit  together  or  that  the  required  production  processes  will  not  be  reproduc¬ 
ible.  In  any  case,  the  government  as  the  design  owner  must  bear  tho  costs  of  making 
whatever  changes  are  necessary  to  produce  a  correct  design.  Changes  under  these 
conditions  can  easily  double  the  costs  of  the  design  development.  If  the  development 
costs  are  a  significant  portion  of  the  total  program  costs  (over  5%),  a  design  valida¬ 
tion  phase  should  be  considered  mandatory.  As  an  additional  note,  changes  which 
occur  during  design  validation  must  themselves  be  proofed  by  unbiased  analysis. 

On  large  quantity  acquisitions,  the  design  documentation  can  be  validated 
through  a  LEADER-FOLLOWER  contractual  team  or  through  other  multisource  pro¬ 
curement  techniques. 

The  other  key  elements  to  acquiring  detailed  design  ownership  are  technically 
knowledgeable  design  management  and  good  configuration  control;  this  is  true 
whether  the  government  or  the  contractor  will  own  the  design.  If  the  government  will 
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assume  ownership,  these  elements  must  reside  within  the  government.  If  the  contrac¬ 
tor  will  retain  ownership,  his  failure  to  provide  these  elements  constitutes  risks  to 
the  project  which  must  be  dealt  with  by  the  government.  The  primary  defense  against 
incurring  excessive  risks  of  this  type  lies  in  the  structuring  of  the  source  evaluation 
criteria,  placing  the  greatest  emphasis  on  technical  expertise  in  both  design  and  pro¬ 
duction  and  significant  emphasis  on  their  configuration  management  system.  Unfor¬ 
tunately,  the  vagaries  of  contracting  will  occasionally  lead  to  an  award  to  a 
contractor  weak  in  one  or  both  areas;  the  only  defense  the  manager  can  provide 
himself  is  a  combination  of  in-house  resources,  effective  management  reports 
(required  through  CDRL),  and  prayer. 

Another  pitfall  exists.  Large  contractors  sometimes  develop  a  design  in  one 
division  and  produce  the  product  in  another  division  located  10  states  away.  Even 
though  the  contractor  will  retain  design  ownership,  the  design  must  be  transitioned 
from  the  one  division  to  the  other,  and  the  separate  divisions  take  on  the  character  of 
different  companies.  To  provide  for  a  good  transition,  the  government  should  require 
that  a  preproduction  model  be  produced,  as  a  part  of  the  development  contract,  with 
the  same  facilities  (and,  preferably,  same  personnel)  which  will  be  used  for  the  pro¬ 
duction  contract. 

As  with  the  system  integration  issue,  when  the  contractor-owned/government- 
owned  design  boundaries  are  established,  they  should  be  clearly  established  and  con¬ 
sistent  with  the  determinations  made  for  the  other  mqjor  issues  and  with  the  criteria 
above. 


SPECIFICATION  TYPE 


Production  items  are  specified  by  product  specifications.  M1L-STD-490  defines 
four  different  types  of  product  specifications  for  use  by  DoD  in  seven  different  for¬ 
mats,  and  while  industry  does  not  necessarily  conform  to  these  formats,  the  different 
types  are  fairly  universal.  The  most  important  types  are  the  functional  (or  perform¬ 
ance)  specification  and  the  fabrication  (or  detailed  design)  specification;  the  remain¬ 
ing  two  typee  are  the  Noncomplex  Item  Product  Fabrication  Specification  (which,  as 
the  name  implies,  is  an  abbreviated  form  of  fabrication  specification)  and  the  Com¬ 
puter  Program  Product  Specification.  Both  the  functional  and  fabrication  types  may 
bo  written  in  two  formats  depending  upon  the  complexity  of  the  item  specified  (i.e., 
prime  item  or  critical  item);  additionally,  the  functional  specification  is  used  in  an¬ 
other  format  as  an  Inventory  Item  Specification.  Which  specification  typo  should  be 
used  is  a  major  issue  which  must  be  resolved  in  selecting  a  procurement  mode. 

FUNCTIONAL  SPECIFICATION 


The  functional  specification  requires  that  the  finished  hardware  meet  size, 
weight,  external  configuration  and  mounting  provisions,  external  interface  require¬ 
ments,  and  overall  performance  of  the  item  within  a  specified  environment  (often  re¬ 
ferred  to  as  “form,  fit,  function,"  or  F3,  parameters).  When  the  contractor  has 
produced  the  hardware,  the  government  is  obligated  to  accept  and  pay  for  it  if  it 
meets  the  specified  requirements,  regardless  of  the  nature  of  the  internal  design.  This 
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approach  is  utilized  frequently  for  the  procurement  of  expendable,  nonrepayable 
items  and  for  readily  available  items  (such  as  batteries)  for  which  the  government  is 
not  concerned  about  internal  configuration.  When  used  to  procure  repairable  items, 
the  procurement  package  should  include  warranty  provisions,  renewable  maintenance 
contract  provisions,  and/or  provisions  for  contractor  services  to  set  up  the  necessary 
government  maintenance  capabilities  to  support  the  equipment  through  its  intended 
service  life. 

The  advantages  of  procurements  controlled  by  functional  specifications  are  as 
follows: 

1.  Detailed  design  responsibility  is  clearly  assigned  to  the  contractor.  If  the 
item  fails  to  meet  specifications,  the  contractor  must  alter  the  design  until 
specified  operation  is  achieved. 

2.  There  is  no  design  data  package  for  the  government  to  procure  or  main¬ 
tain. 

3.  Requirements  for  technical  capability  within  the  government  are  mini¬ 
mized.  This  is  the  path  of  least  involvement  on  the  part  of  the  government 
in  contracting,  contract  monitoring,  etc. 

4.  Standardization  can  be  achieved  among  multiple  sources  through  two-way 
interchangeability  of  products  which  may  differ  internally.  These  multiple 
sources  may  be  exercised  simultaneously. 

The  disadvantages  include  the  following: 

1.  Each  procurement  contains  a  development  effort  unless  the  product  is  off- 
the-shelf-unmodified.  Some  time  and  money  are  involved  each  time  the 
item  is  procured  for  engineering,  changes,  production  learning  curves,  and 
debugging.  Since  the  contractor  develops  the  product,  companies  without  a 
development  capability  (and  accompanying  development  overhead)  are  ex¬ 
cluded.  (Of  course,  when  an  off-the-shelf  product  is  being  purchased  un¬ 
modified  or  slightly  modified,  the  development  costs  (apportioned  into  each 
unit  price)  are  shared  by  the  entire  market,  of  which  the  government  may 
be  only  a  small  part.) 

2.  Each  time  a  procurement  is  made,  the  contractor  who  has  the  least  appre¬ 
ciation  for  the  total  significance  of  the  specification  and  the  effort  to  ac¬ 
complish  the  task  is  most  likely  to  be  the  low  bidder.  This  means  that  the 
source  selection  criteria  must  be  very  carefully  constructed  to  include 
mechanisms  to  demonstrate  contractor  awareness  of  critical  elements  as 
well  as  the  capabilities  to  produce  the  item.  In  a  simultaneous  multisource 
procurement,  the  quantities  to  each  successful  bidder  can  be  made  contin¬ 
gent  (within  boundaries  or  as  determined  by  a  formula)  upon  the  relative 
performance  of  each  product  in  preproduction  testing  for  a  limited  set  of 
critical  parameters  (such  as  receiver  sensitivity,  reliability,  and  cost). 

3.  The  costs  of  repair  parts  will  tend  to  become  excessive  when  a  contractor 
realizes  that  he  is  in  a  somewhat  sole-source  position  with  respect  to  his 
equipment  unless  the  total  maintenance  for  the  service  life  of  the 
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equipment  is  provided  for  in  the  procurement  contract  or  in  conjunction 
with  the  procurement  contract  while  competition  is  still  being  maintained. 
A  side  issue  arises  in  that  procurement  funds  and  operations/maintenance 
funds  are  separate  budget  pots,  requiring  the  program  to  resolve  a  poten¬ 
tially  politically  sensitive  issue  on  the  funding  of  the  procurement  prior  to 
its  execution. 

4.  Careful  specification  of  all  external  parameters  is  required  to  ensure  true 
interchangeability.  Each  parameter  must  be  specified  within  tolerances;  it 
is  strongly  recommended  that  output  tolerances  be  tighter  than  the  input 
tolerances  they  interface  with  and  inversely  for  input  tolerances.  Control 
second-and  third-order  parameters  (timing  relationships,  phase,  out-band 
response,  spurious  emission,  etc.);  overlooked  second-  and  third-order 
parameters  may  result  in  marginal  or  unusable  equipment  that  the  govern¬ 
ment  is  obligated  to  accept.  Considerable  testing  may  be  required  to  derive 
this  information. 

Even  if  the  functional  specification  is  for  a  less  complex  item  than  a  major  system, 
the  specification  guidance  provided  for  Type  Cla,  MIL-STD-490,  App  VII,  contains 
the  elements  which  should  be  considered  even  if  another  format  is  used.  There  are 
several  points  which  may  be  clarified  for  functional  specifications.  The  following 
items  amplify  type  Cla  paragraph  guidance: 

1.  Maintainability.  Consider  test  equipment/test  point  provisions  and  any 
special  requirements  such  as  use  of  ATE  or  limits  to  GPETE  and  the  inter¬ 
faces  to  be  provided  for  test  purposes, 

2.  Design  and  Construction.  Specifically  applicable  paragraphs  of  the  general 
equipment  specifications  (MILrE-16400,  MIL-E-5400  etc.)  should  be  cited. 
Metric/English  requirements  should  be  called  out. 

3.  Materials.  Processes,  and  Parts.  Include  the  provisions  to  (a)  prevent  the 
unnecessary  use  of  strategic  and  critical  materials,  (b)  prevent  the  use  of 
processes  which  are  known  to  lead  to  an  unsafe  item  in  field  use,  and  (c) 
limit  the  use  of  proprietary  or  other  nonstandard  parts  as  appropriate  to 
the  maintenance/logistic  support  plan  for  the  item. 

4.  Interchangeability.  Applicable  items  include  type  and  location  of  connec¬ 
tors,  connector  pin  allocations,  and  interface  hardware  for  mounting,  far 
cooling  connections,  for  power,  for  insert  keying  configurations,  etc. 

5.  Human  Performance/Hiiman  Engineering.  Drawings  of  specific  control 
panel  configurations,  control  operations,  and  lighting  features  should  be 
referenced  when  appropriate. 

FABRICATION  SPECIFICATION 


The  fabrication  specification  requires  the  hardware  to  be  built  in  accordance 
with  a  detailed  design  data  package.  In  this  manner,  “form”  and  “fit”  are  tightly  con¬ 
trolled  and  “function”  is  implicitly  controlled  by  the  design  capabilities  of  the  hard¬ 
ware  described  by  the  data  package.  Only  critical  functional  parameters  are  include  4 
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to  ensure  that  the  accepted  hardware  will  be  functional.  A  fabrication  specification  is 
always  used  for  government-owned  detailed  designs  (and  vice  versa)  (See  DESIGN 
OWNERSHIP,  above).  The  advantages  of  fabrication  specification  controlled  procure¬ 
ments  are: 

1.  The  government  maintains  strict  configuration  control;  thus,  all  parts  in 
the  field  are  identical  to  their  counterparts  regardless  of  manufacturer 
(unless  specifically  authorized  changes  are  allowed).  Thus,  repair  parts, 
training,  test  equipment,  technical  manuals,  and  other  logistics  elements 
be  standardized  and  maintained  efficiently  and  cost-effectively  at  the 
organizational  or  intermediate  maintenance  levels. 

2.  The  development  costs  and  associated  long  lead  times  are  borne  only  once 
and  the  government  can  exercise  strict  configuration  management  during 
design.  The  design  can  be  fabricated  by  any  contractor  with  the  proper 
production  facilities  and  competence  to  use  them.  This  is  a  much  broader 
source  base  than  that  for  functional  specifications.  Also,  contractors  main¬ 
taining  production  facilities  only  do  not  have  the  high  overhead  rate  asso¬ 
ciated  with  a  development  capability  (usually  an  acceleration  of  20-30%). 

3.  Second-  and  third-order  parameters  which  are  inherent  in  the  specified 
design  will  continue  to  play  an  impoitant  but  unknown,  unappreciated, 
and  unspecified  role  in  the  successful  operation  of  the  hardware. 

4.  Lessons  learned  in  the  production  of  the  item  by  one  contractor  may  be 
incorporated  in  the  data  package  to  preclude  duplicate  difficulties  on  suc¬ 
ceeding  contracts,  thus  reducing  risks  with  each  procurement. 

5.  Spurious  reprocurements  and  mobilization  requirements  can  be  met  rap¬ 
idly  and  without  significant  risk. 

G.  Good  cost,  quality,  and  production  time  standards  can  be  obtained  and 
utilized  on  future  procurements. 

7.  The  in-house  technical  base  is  generally  increased  and  made  available  for 
future  procurements. 

Most  of  the  disadvantages  of  fabrication  specification  procurements  relate  to 
the  fact  that  the  government  owns  the  detailed  design  and  must  bear  the  responsibili¬ 
ties  of  ownership  (see  DESIGN  OWNERSHIP,  above).  Some  other  disadvantages 
include: 

1.  The  government  must  pay  for  all  development,  validation,  and  qualifica¬ 
tion  costs  incurred  in  assembling  the  design  data  package. 

2.  The  original  developer  must  be  honest,  accurate,  complete,  and  current  in 
his  generation  of  the  design  data  package.  A  lack  of  one  of  these  character¬ 
istics  will  show  up  in  a  design  validation  phase;  however,  complete  valida¬ 
tion  is  frequently  omitted  (too  expensive).  Even  when  validation  is 
performed,  large  costs  will  be  incurred  correcting  deficiencies. 

3.  The  state  of  the  technology  base  is  inherent  in  the  data  package.  Highly 
mobile  technologies  can  rapidly  outdate  the  design  and  cause  costly,  hard- 
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to-get  spare  parts  sin  d/or  costly  redesign  of  the  equipment.  It  is  very  diffi¬ 
cult  to  accommodate  new  capabilities  without  costly  design  changes. 

4.  Because  functional  parameters  are  largely  not  available,  it  is  very  difficult 
to  create  a  standardization  program  which  utilize  the  item  in  ever-expand¬ 
ing  applications  and  updates  the  capabilities  to  conform  to  dynamic  opera¬ 
tional  needs. 

SELECTION  CRITERIA 

The  choice  between  specification  types  to  support  procurement  is  based  on: 

1.  The  need  for  detailed  design  ownership.  (Detailed  design  ownership  implies 
fabrication  specification.) 

2.  The  size  of  procurement  and  reprocurement  requirements.  (For.  items  re¬ 
quiring  development,  functional  specifications  are  best  applied  to  large 
procurements  where  multisources  can  be  utilized  simultaneously;  fabrica¬ 
tion  specifications  are  valuable  to  support  intermittent  and  spurious 
reprocurement  requirements,) 

3.  The  maturity  of  product  design.  (Functional  specifications  are  best  applied 
to  mature  (on  shelf)  designs  which  can  be  used  as  is  or  with  minor  modifi¬ 
cations.) 

In  cither  case,  product  specifications  are  intended  for  use  with  fixed-price  contracts. 

If  the  item  is  repairable,  the  maintenance  of  items  procured  on  functional  specifica¬ 
tions  should  be  included  in  the  contract;  when  this  is  not  possible,  a  fabrication  speci¬ 
fication  may  be  a  better  choice. 

Picking  one  specification  type  over  the  other  has  long-term  effects  which  must 
be  dealt  with.  Fabrication  specifications  are  effective  establishing  standardization 
and  configuration  control  only  at  the  piece  part  level;  this  fact  may  be  inconsistent 
with  the  resolution  of  the  standardization  and  level-of-repair  issues.  Standardization 
above  the  piece  part  level  is  best  supported  by  functional  specifications;  therefore 
functional  specifications  should  be  established  at  each  level  of  complexity 
down  to  the  level  of  standardization  regardless  of  the  specification  type 
used  for  procurement. 

Occasionally,  procurements  are  observed  in  which  the  government  owns  a  de¬ 
tailed  design  data  package  but  lacks  confidence  in  the  accuracy  and  completeness  of 
the  design.  It  has  been  common  practice  to  furnish  the  drawings  the  contractor  “for 
information  only,”  and  require  them  to  retain  interchangeability  with  depicted  design 
to  a  specified  level.  This  approach  amounts  to  procurement  without  specification, 
since  functional  specification  is  not  established  and  the  government  disclaims  respon¬ 
sibility  for  the  adequacy  of  the  baseline  designs.  The  contractor  assumes  high  risks 
which  are  translated  into  virtually  certain  failure  to  meet  technical,  cost,  or  schedule 
goals.  None  of  tho  advantages  of  cither  specification  type  are  realized. 

The  application  of  functional  specifications  to  procurements  of  small-lot, 
government-peculiar  items  requiring  significant  development  is  observed,  also. 
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Primarily,  the  manager  is  attempting  to  procure  the  item  without  paying  for  the 
design  package.  This  ploy  works  until  a  reprocurement  of  even  one  more  item  is 
required;  then  the  contractor,  who  is  in  a  sole-source  position,  must  be  paid  again  for 
development,  tooling,  etc.  Usually,  the  procurement  situation  arises  out  of  an  expen¬ 
sive  and  complex  development  effort  in  which  small  quantities  are  procured  initially 
but  large  quantities  are  eventually  required  —  but  in  a  number  of  small-quantity 
reprocurements.  In  this  situation,  the  government  should  buy  reprocurement  rights 
with  the  initial  procurement  which  guarantee  that  future  procurements  will  be  met 
at  the  same  unit  cost  (plus  escalation  for  inflation)  as  the  first  units  and  within  a 
specified  time.  In  this  way,  the  government  pays  the  contractor  to  maintain  the 
design  documentation  and  production  tooling  until  it  is  needed.  Essentially,  the  gov¬ 
ernment  has  bought  use  of  the  design  without  obtaining  title  for  custody  to  the  data 
package.  This  ploy  is  not  nearly  as  flexible  as  obtaining  design  ownership  out  of  the 
development  phase  and  is  harder  to  implement  on  a  firm  legal  basis,  but  costs  can  be 
significantly  less  in  development.  The  tactic  may  not  be  used  for  process-sensitive 
procurements  because  the  process  yield  will  be  lost  between  buys. 

Functional  specification  implies  contractor-owned  detailed  design,  and  fabrica¬ 
tion  specification  implies  government-owned  detailed  design.  This  is  fine,  but  who 
should  be  responsible  for  developing  the  specified  parameters?  In  functional  specifica¬ 
tions,  the  parameters  are  generated  by  the  government  from  the  requirements;  addi¬ 
tional  functional  parameters  may  be  added  to  extend  the  specification  to  a  lower  level 
of  complexity  by  adopting  established  standard  interfaces  or  by  procuring  additional 
functional  specification  parameters  from  the  contractor  and  verifying  them  during 
qualification  tests.  This  latter  ploy  is  slightly  more  risky  because  second-  and  third- 
order  parameters  are  easier  to  overlook.  Fabrication  design  data  development  is  very 
much  more  complicated  and  controversial. 

The  final  subissue  related  to  specification  type  involves  the  level  of  specifica¬ 
tion;  i.e.,  the  level  of  complexity  to  which  the  system  is  specified.  Functional  speci  - 
cations  should  be  established  to  the  lowest  level  of  complexity  consistent  with  the 
other  mqjor  issue  resolutions;  fabrication  specifications  should  be  established  consis¬ 
tent  with  the  procurement  considerations  and  design  ownership  boundaries.  The 
specifications  should  be  complete  to  the  design  ownership  boundaries  prior  to  start¬ 
ing  procurement  actions. 

LEVEL  OF  REPAIR  PERFORMED  AT 
THE  FIELD  LEVEL 


The  issue  of  level  of  repair  performed  at  the  field  level  should  be  resolved 
before  the  transition  to  production  is  commenced.  The  field  maintenance  level 
includes  the  organizational  maintenance  leveis  and  the  field/afloat  intermediate 
maintenance  level.  The  level  of  repair  (LOR)  —  i.e.,  the  level  of  system  complexity  to 
which  the  system  is  repaired  —  performed  at  the  organizational  level  should  bo  deter¬ 
mined  during  concept  formulation  in  conjunction  with  level  of  maintenance  capability 
constraints.  These  constraints  tend  to  force  the  LOR  at  the  organizational  level 
toward  the  system  level  of  complexity;  however,  practical  constraints  (size,  weight, 
item  cost,  and  failure  rate)  tend  to  drive  the  LOR  toward  the  piece  part  level.  The 
balance  of  these  forces  tends  to  resolve  the  system  maintenance  philosophy  at 
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approximately  the  unit  (or  WRA,  level  of  complexity.  However,  the  best  level  at 
which  to  discard  failed  items  is  usually  simpler  (such  as  the  module  or  piece  part 
level)  than  the  level  of  organic  repair.  In  order  to  minimize  downtime  and  inventory 
quantities,  some  form  of  intermediate  maintenance  activity  is  usually  employed  to 
effect  repair  of  the  unit  at  a  SRA  level  and/or  to  establish  a  rotatable  equipment  pool. 
A  combination  of  noneconomic  analysis  and  an  initial  support  analysis  based  in  the 
system  life-cycle  cost  model  is  completed  to  determine  the  level  of  repair  at  the  inter¬ 
mediate  activities.  After  the  product  design  is  relatively  complete,  a  level  of  repair / 
logistics  support  analysis  performed  as  a  portion  of  the  ILS  tasks  is  completed  to 
confirm  and  modify  appropriately  the  initial  determination. 

Although  tedious,  the  economic  analysis  is  relatively  straightforward  if  a 
viable  system  life-cycle  cost  model  has  been  established;  guidance  is  available  from 
MIL-STD-1390,  MIL-STD-1388,  and  system  effectiveness  personnel  (such  as  those  in 
NOSC’g  Product  Assurance  Division).  Noneconomic  analysis  may  or  may  not  play  a 
significant  role  in  the  LOR  decision. 

The  noneconomic  analysis  will  consist  of  recognizing  preempting  factors  such 
as  safety,  repair  feasibility,  standardization,  allowable  downtime,  and  the  other 
noncost  constraining  factors.  If  this  analysis  produces  a  definitive  decision  for  LOR, 
the  economic  analysis  is  still  completed  to  formulate  repair/discard  decisions  on  the 
basis  of  the  design.  Usually  a  definitive  decision  will  not  be  reached  until  the  product 
design  is  nearly  complete.  Nevertheless,  factors  which  affect  the  noneconomic  analy¬ 
sis  do  impact  upon  the  build,  buy,  modify  alternatives,  the  system  integration  issue, 
and  the  ability  to  achieve  service  approval.  Of  these  factors,  allowable  downtime  and 
standardization  considerations  tend  to  be  the  confusing  factors.  Standardization  is 
itself  a  nuyor  issue  (see  STANDARDIZATION,  below);  downtime  is  discussed  in  the 
following  paragraphs. 

The  downtime  allowable  for  the  system  as  a  whole  is  determined  by  its 
criticality  to  the  platform  missions;  the  system  is  classified  as  vital,  semivital,  or  non- 
vital  with  a  system  availability  assigned  accordingly.  Downtime  and  mission  reliabil¬ 
ity  are  the  factors  which  make  up  availability;  the  formulation  normally  is  of  the 
form 


MTBF 

'  MTBF  +  MDT 

Not  all  failures  which  can  occur  affect  mission  eligibility,  and  the  repair  of  those  fail¬ 
ures  does  necessarily  cause  the  system  to  be  down.  Prior  to  the  product  design  phase, 
essential  equipment  items  (those  whose  failure  effects  mission  reliability)  can  be  iden¬ 
tified  at  each  level  of  complexity  within  the  system.  When  the  system  is  partitioned, 
many  (ideally  all)  of  the  essential  items  can  be  lumped  into  a  single  cell,  and  at  some 
level  of  complexity  it  is  possible  to  eliminate  the  essential  item  nature  through  voting 
techniques,  redundancy,  etc.  Unfortunately,  it  is  not  always  practical,  even  when  it  is 
cost-effective,  to  partition  in  this  way,  since  other  factors  come  into  play.  However,  a 
level  of  complexity  can  be  identified  for  each  portion  of  the  system  for  which  essential 
item  effects  are  minimal  and  for  which  the  allowable  downtime  can  bo  notably  long 
without  affecting  system  availability.  This  level  allows  effective  intermediate  mainte¬ 
nance  support  with  practical  turnaround  times  and  is,  therefore,  the  optimum  level  of 
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repair  from  a  downtime  viewpoint.  Naturally,  the  final  LOR  determinations  must 
take  into  account  many  other  factors. 

When  the  level  of  repair  performed  at  the  field  level  is  identified,  the  informa¬ 
tion  is  useful  in  establishing  tradeoffs  between  contractor  support  versus  in-house 
support,  government  configuration  control  requirements,  and  other  factors  important 
to  resolving  the  other  mqjor  issues. 

STANDARDIZATION 


The  values,  policies,  and  tradeoffs  associated  with  standardization  are  docu¬ 
mented  in  chapter  XIV,  Consideration  of  Standardization.  Level  of  standardization  is 
a  concept  which  recognizes  the  benefits  of  standardization  while  acknowledging  that 
improper  implementation  can  lead  to  gross  deficiencies  that  negate  the  potential 
benefits.  For  a  given  operational  requirement,  it  is  beneficial  to  have  standard  sys¬ 
tem,  design,  logistics,  etc.,  to  meet  the  requirement.  On  the  other  hand,  logistics 
items  (piece  parts,  modules,  etc.)  must  be  utilized  in  many  systems  to  become  effec¬ 
tive  standard  items.  System  designers  require  flexibility  and  acquisition  managers 
require  flexibility,  but  standardization  implies  inflexibility.  In  order  to  avoid  inflexi¬ 
bility  and  to  be  useful,  standards  must  be 

•  Functionally  specified  (so  interface  data  are  readily  available) 

•  Functionally  complete  (can  be  used  as  a  building  block) 

•  Available  in  minimal  variety  sufficient  to  meet  differing  mqjor  applications 

•  Adaptable  (easily  maintained  with  the  state  of  the  art) 

•  Well  documented  and  readily  available  to  the  prospective  user 

•  Functionally  flexible  (possess  capabilities,  reliability,  etc.,  that  make  it 
attractive  to  diverse  applications) 

Standardization  has  strong  implications  for  each  of  the  other  nifyor  issues. 
System  partitioning  must  be  carefully  constructed  to  use  existing  standards  where 
possible  and  to  make  other  partitioned  portions  of  the  system  attractive  as  new  stan¬ 
dards.  The  partitions  should,  as  much  as  possible,  conform  to  industry  standards 
(where  they  exist),  common  practice  etc.,  to  avoid  “swimming  upstream”  and  make 
the  item  easier  to  procure.  The  “standard”  partitions  should  be  consistent  with  design 
ownership  and  levcl-of-repair  decisions.  System  partitioning  which  is  system  peculiar 
(unsuitable  for  other  applications)  should  be  confined  to  ns  small  a  portion  of  the  sys¬ 
tem  as  is  practical;  this  puts  limitations  on  other  partitioning  considerations  such  as 
those  for  level  of  repair. 

Functional  specifications  should  be  established  at  each  level  of  complexity  be¬ 
low  the  system  level  and  above  (but  including)  the  level  of  standardization.  The  level 
of  standardization  will  be  the  worst  case  of  the  design  ownership  boundary  and  the 
level  of  ropair.  The  system-peculiar  items  should  be  specified  also,  since  future  efforts 
to  standardize  or  to  change  an  interfacing  standard  may  have  to  identify  critical 
interface  parameters. 


VI-17 


MAJOR  ISSUES  SUMMARY 


Each  major  issue  embodies  a  family  of  often  sensitive  considerations  which 
must  be  dealt  with  before  a  successful  transition  to  production  can  be  completed. 
These  issues  are  closely  related  and  may  not  be  considered  as  isolated  cases  without 
risk  of  causing  dire  perturbations  to  the  others.  Many  of  the  steps  which  must  be 
taken  to  resolve  (or  resulting  from)  these  issues  involve  time  and  money  resources 
which  may  appear  to  be  unnecessary  when  viewed  in  isolation;  however,  failure  to 
execute  these  steps  within  the  coordinated  framework  of  the  mqjor  issues  will  lead  to 
greater  overall  program  costs. 

THE  “BUY”  ALTERNATIVE 


The  buy  alternative  describes  the  use  of  existing  equipments  and  presumes 
that  the  equipments  are  readily  available  and  in  use,  and  that  some  use  history  might 
be  obtainable.  Such  equipments  will  be  identified  by  the  source  search  and  screen 
approach  integral  to  the  TELCAM  technique  (see  chapter  XI,  Screening  Techniques). 
This  screening  serves  to  establish  qualified  equipments  but  does  not  qualify  the 
equipment  for  service  use  directly;  rather,  the  equipments  must  be  integrated  into  the 
system  and  the  system  support  plan  modified  as  necessary  to  accommodate  the  item 
(the  modifications  must  remain  within  the  system  constraints).  For  the  portions  of 
the  system  utilizing  this  alternative,  the  mqjor  issues  will  be  resolved  as  follows: 

Commercial  Equipment  Military  Equipment 


System  Integration 
Design  Ownership 
Specification  Type 

LOR 


Standardization 

“As  previously  established”  implies  that  this  information  is  available  and  compatible 
v/ith  system  requirements;  incompatibilities  may  often  be  resolved  by  treating  the 
military  equipment  as  a  commercial  equipment.  The  buy  alternative  flow  is  illus¬ 
trated  in  figure  VI-3. 

A  sophisticated  “buy"  technique  which  is  suitable  for  large,  multiple-buy 
procurements  (such  as  avionics)  is  documented  in  ARINC  Publication  1313-01-1- 
1447,  Application  of  the  Commercial  Airline  Acquisition  Methodology  to  Department 
of  the  Navy  Electronic  Equipment  Acquisitions,  15  October  1975,  AD/A015694. 


Government  responsibility 

Supplier 

Functional 

At  the  equipment  level 
At  the  equipment  level 


from  the  equipment  level  on  up 

As  previously  established 

Inventory  item 
(functional) 

No  lower  than  that  pre¬ 
viously  established 

As  previously  established 


VI-18 


THE  “MODIFY”  ALTERNATIVE 


The  modify  alternative  is  usable  when  modifications  can  make  an  existing 
equipment  suitable  for  the  system  application.  The  same  steps  and  procedures  apply 
as  for  the  buy  alternative  except  that  a  modification  step  is  added.  The  modification 
may  be  performed  by  the  government  or  by  the  supplier  for  commercial  equipments 
depending  upon  the  support  provisions  in  the  procurement  contracts  and  the  nature 
of  the  modifications.  Modifications  which  can  be  accomplished  externally  to  the 
equipment  in  the  interfacing  hardware  will  normally  be  done  by  the  government;  all 
other  modifications  are  usually  accomplished  by  the  supplier.  The  modification  is 
controlled  by  detailed  design  when  accomplished  by  the  government  and  by  incorpo¬ 
ration  into  the  procurement  specification  when  done  by  the  supplier.  The  two  modifi¬ 
cation  alternative  flows  are  illustrated  in  figure  VI-3. 

THE  “BUILD”  ALTERNATIVE  (DEVELOPMENT) 


The  build  alternative  should  be  pursued  only  when  suitable  buy  and  modify 
alternatives  are  not  available.  Unfortunately,  the  many  unique  military  requirements 
and  rapidly  evolving  military  technology  force  a  development  situation;  even  where 
otherwise  suitable  equipment  exist,  the  requirement  maintenance  environment  can¬ 
not  be  adapted  to  it. 

Development  is  a  very  much  more  complex  process  than  the  processes  of  the 
buy  and  modify  alternatives;  therefore,  there  are  more  issues  to  be  resolved,  moro 
things  that  can  go  wrong,  more  resources  required,  more  time  needed,  and  moro  risks 
which  must  be  assumed  and  managed.  Whereas  each  of  the  major  issues  is  largely 
resolved  for  the  other  alternatives,  tho  resolution  of  these  issues  is  not  automatic  and 
can  be  critical  to  the  success  of  the  build  alternative;  also,  the  complexity  of  develop¬ 
ment  includes  some  issues  peculiar  to  the  process.  These  factors  may  make  build  the 
least  desirable  alternative.  It  is  necessary  to  separate  requirements  from  desirements, 
to  ensure  requirements  validity,  and  to  research  possible  alternatives  which  can  lead 
to  a  buy  or  modify  decision  if  the  time,  expense,  and  risk  of  development  are  to  be 
avoided;  a  strong  validation  phase  is  very  important  for  this  reason  (see  chapter  V, 
Validation  Phase).  The  purpose  of  the  transition  to  production  is  to  obtain  a  product 
for  application  to  service  needs;  expansion  of  the  technology  base  is  an  exploratory 
development  function  which  only  increases  the  risks  incurred  when  accomplished 
during  full-scale  development. 

Before  entering  into  development,  the  system  must  be  partitioned:  this  should 
normally  bfl-done.in  such  a  wav  that  portions  subject  to  the  buy  or  modify  alterna¬ 
tives  can  be  independent  of  the  developed  portions  as  much  as  possible.  Partitioning 
along  such  lines  is  normally  compatible  with  the  resolution  of  the  major  issues;  but 
should  conflict  occur,  partitioning  to  segregate  the  buy/modify  portions  from  the 
build  portions  should  normally  be  given  precedence  unless  the  system  life-cycle  cost 
model  proves  that  development  of  the  portion  in  question  is  a  cheaper  alternative 
(although  rare,  the  situation  can  occur).  Tho  remainder  of  this  section  deals  exclu¬ 
sively  with  development. 
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Figure  VI-3.  "Buy’  and  'modify'  flow. 


There  are  two  issues  peculiar  to  development  (in  addition  to  the  five  mqjor 
issues  above): 

•  The  development  alternative  to  pursue 

•  The  transition-to-industry  point 

The  issues  are  interrelated  and  are  primarily  related  to  the  resolution  of  the  system 
integration  and  design  ownership  issues.  The  level  of  repair  should  be  resolved  at  the 
most  cost-effective  level  at  which  it  is  feasible  within  the  mission  constraints.  The 
specification  type  will  be  dictated  by  design  ownership  and  the  selected  procurement 
mode,  and  the  standardization  issue  will  be  determined  by  the  system  partitioning, 
design  ownership,  and  level-of-rcpair  issues.  The  selection  of  the  development  alterna¬ 
tive  should  be  based  primarily  on  the  management  of  perceived  risks  and  on  the 
maintenance  of  competition  (when  the  development  is  accomplished  by  industry); 
however,  the  decision  may  bo  modulated  by  tho  other  issues.  Transition  to  industry 
embodies  these  considerations  plus  a  great  deal  of  controversy  and  political  risk. 

Only  under  the  special  circumstances  outlined  in  chapter  XVII,  Development 
Alternatives,  can  a  development  be  pursued  in-house;  these  same  criteria  may  bo 
applied  to  the  production  phase  with  the  result  that  industry  will  almost  always  bo 
tho  source  of  production  resources.  Onco  tho  transition  to  industry  has  taken  place, 
tho  reverse  transition  should  not  occur  unless  now  production  resources  must  bo 
recruited.  Usually,  the  reverse  transition  is  only  a  temporary  one  to  effect  transition 
from  a  sole  source  to  a  multisourco  situation  (ref  NAFI TR-1901);  a  permanent 
reverse  transition  normally  occurs  only  when  the  government  is  dependent  on  an 
item  in  which  competent  industry  sources  aro  no  longer  interested. 

Tho  revorso  transition  serves  to  develop  a  data  package  in  which  the  govern¬ 
ment  has  high  confidence;  it  can  bo  avoided  if  proper  care  is  taken  in  procuring  ade¬ 
quate  documentation  initially. 

The  transition  may  occur  at  tho  beginning  of  the  acquisition  cycle  as  in  the 
case  in  which  a  system  is  wholly  dependent  on  a  technology  which  has  beer,  developed 
within  industry,  or  it  may  occur  following  pilot  production  (and  coinciding  with  the 
release  to  production  decision).  Or  both  transitions  may  take  place  with  a  reverse 
transition  occurring  after  engineering  development.  Other  transition  points  are  possi¬ 
ble  (see  fig.  VI-4). 

If  the  design  ownership  is  going  to  reside  with  industry,  the  transition  to  in¬ 
dustry  point  should  depend  on  tho  need  for  competition  in  the  procurement  mode  and 
the  maturity  of  the  design.  If  the  design  constitutes  a  mqjor  modification  of  commer¬ 
cial  products  and  does  not  depend  on  new  technology,  competition  is  relatively  inex¬ 
pensive  to  achieve.  However,  if  the  design  depends  heavily  on  new  technology  or  on 
technologies  not  commonly  used  in  established  largo-market  products,  competition 
will  be  limited  by  program  resources  because  a  large  amount  of  high-risk  development 
will  be  duplicated  by  as  many  sources  as  are  needed  to  support  competition  (at  least 
two  —  more  if  the  technical  risk  is  considered  high).  Competition  is  important  to 
reduce  cost/schedule  risks  and  to  maintain  control  of  the  procurement.  The  program 
must  have  the  resources  and  justification  based  on  usage  to  support  these  conditions; 
otherwise,  the  design  ownership  should  reside  with  the  government. 
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4  POSSIBLE  TRANSITION  TO  INDUSTRY 
SEE  TEXT  AND  CHAPTER  XVII 
#  REVERSE  TRANSITION  TO  IN-HOUSE 


Figure  VI -4.  “Build"  (develop)  flow. 
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In-house  resources  must  normally  be  used  to  develop  adequate  operational  and 
technical  requirements  for  the  specification  used  to  document  the  transition  to  indus¬ 
try. 


Mo6t  frequently,  the  conditions  which  force  the  program  into  development  will 
also  favor  the  government-owned  detailed  design  decision  which  is  supported  by  a 
fabrication-type  product  specification.  The  transition  to  industry  point  will  depend 
initially  on  whether  in-house  resources  or  industry  sources  are  to  be  utilized  for 
development  (see  the  discussion  below).  In  either  case,  a  competitive  development 
may  be  indicated  to  choose  between  different  technical  approaches  which  are  feasible. 
If  industry  has  developed  a  technology  on  which  the  equipment  is  primarily  depend¬ 
ent,  the  same  industry  sources  will  probably  be  used  through  engineering  develop¬ 
ment.  If  in-house  resources  have  been  utilized  through  the  proof  of  feasibility 
(advanced  development),  a  probable  transition  point  is  at  the  entry  into  engineering 
development. 

The  primary  purposes  of  engineering  development  are  product  design  and  pro¬ 
duction  engineering  i.e.,  those  tasks  which  make  an  item  readily  reproducible.  The 
functional  design  of  the  equipment  should  be  complete  within  a  high  confidence  level, 
although  it  is  necessary  for  all  portions  of  the  design  to  lave  been  reduced  to  hard¬ 
ware  and  software.  Therefore,  some  low-risk  design  tasks  may  be  incorporated  into 
engineering  development  in  addition  to  the  inherent  design  tasks;  furthermore,  tasks 
involving  any  computer  software  associated  with  the  equipment  must  be  integrated 
into  the  ED  effort.  Also,  the  EDM  is  the  first  level  at  which  the  design  is  sufficiently 
mature  that  the  ILS  tasks  can  be  completed.  The  coordination  of  these  efforts 

the  system  integration  function  is  a  primary  means  of  coordination  and  control. 

Many  instances  of  serious  problems  in  engineering  development  can  be  cited. 
While  failure  to  meet  technical,  cost,  and  schedule  targets  is  the  result,  the  root  prob¬ 
lem  can  usually  be  traced  to  one  or  more  of  the  following  actions: 

1.  Proceeding  into  engineering  development  without  a  complete  functional 
design  of  a  high  confidence  level;  this  constitutes  an  incomplete  validation 
of  the  technical  approach, 

2.  Failure  to  implement  an  effective  configuration  control  program. 

3.  An  attempt  by  the  government  to  contract  system  engineering/integration 
responsibilities  (at  the  system  level,  this  is  virtually  impossible  for  a  con¬ 
tractor  to  perform  and  results  in  not  meeting  the  responsibilities  at  all). 
Commonly,  many  GFE  items  will  be  included,  each  with  its  own  schedule 
risk,  thus  greatly  compounding  the  program  risk  by  instituting  a  large 
number  of  unnecessary  interdependent  risk  elements.  A  failure  in  one 
affects  the  entire  system. 

4.  Failure  of  the  government  to  procure  and  validate  the  required  design 
data. 

5.  Failure  to  properly  partition  and  specify  the  system  to  the  required  level  of 
complexity  within  elements  contracted  to  industry  (specification  is 
required  to  at  least  the  intended  level  of  repair). 
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6.  Failure  to  identify  and  support  the  in-house  technical  talent  required  to 
monitor  and  control  contractual  development  efforts. 

7.  Failure  to  implement  project  controls  to  allow  knowledgeable  government 
management  team  members  to  make  decisions  within  the  government’s 
system  design/integration  areas  of  responsibility. 

An  effort  to  reduce  costs  usually  arises  from  a  failure  to  estimate  and  budget 
adequately  at  the  inception  of  the  program.  Later,  as  the  program  approaches  the  ED 
phase,  accurate  estimates  are  viewed  as  “cost  overruns*  by  budgeteers.  Once  a  grossly 
inadequate  estimate  is  made  and  accepted,  a  large  risk  of  program  cancellation  exists, 
and  it  grows  with  time;  the  situation  can  often  resolved  only  by  very  strong  justifica¬ 
tions  including  a  detailed  comparison  of  both  estimates  and  by  presenting  a  program 
schedule  which  will  not  impact  the  budget  figures  in  the  next  3  fiscal  years.  On  the 
other  hand,  gross  overestimates  at  program  inception  will  normally  result  in  not 
establishing  a  program  at  all  because  relatively  less  justification  is  available  so  early. 
Unless  detailed,  high-confidence  estimates  can  be  made,  it  is  better  to  make  no  esti¬ 
mate  at  all  or  to  make  highly  qualified  estimates  in  conjunction  with  a  program  plan 
which  allows  for  budget  revisions.  Attempts  to  cut  costs  by  excising  such  “luxuries" 
as  in-house  technical  support  only  eliminate  the  tools  necessary  to  manage  and 
manipulate  the  risks  present  in  any  program.  Without  these  tools,  risks  would 
become  interdependent,  and  a  failure  in  one  risk  element  would  create  a  chain  reac¬ 
tion  of  failures. 

The  in-house  versus  contractor  development  controversy  has  its  basis  in  the 
national  policy  that  the  government  does  not  compete  with  industry.  Therefore,  even 
when  in-house  resources  are  capable  of  handling  the  task  at  costs  below  those  of 
industry,  industry  must  be  selected.  However,  in-house  resources  are  justified  when 
the  ability  to  meet  requirements  is  jeopardized  by  any  industrial  alternative.  It  is 
important  to  realize  that  advances  in  the  state  of  the  art  are  not  sought  during  the 
product  development  phase;  these  are  accomplished  through  exploratory  development. 
The  following  conditions  comprise  valid  justifications  for  in-house  development: 

1.  No  qualified  industrial  interest  expressed  in  the  development. 

2.  The  government  cannot  develop  specifications  adequate  for  contract  and 
cannot  otherwise  obtain  qualified  contract  services  within  the  required 
time  frame. 

3.  The  system  characteristics  and  development  circumstances  involve  a  spe¬ 
cial  case  in  which  in-house  expertise  far  exceeds  industrial  expertise.  This 
special  in-house  expertise  may  involve  technical  requirements  which  are 
closely  related  to  volatile  user  requirements  or  service  doctrine;  also,  spe¬ 
cial  security  requirements  may  be  exclusively  available  to  in-house  exper¬ 
tise. 

NOTE:  In-house  development  does  not  preclude  direct  industrial  support; 
however,  the  direct  responsibility  for  detailed  design  decisions  is  retained  by  the  gov¬ 
ernment.  Direct  contractor  participation  is  encouraged  for  developments  involving 
high-volume  production. 

Whether  the  design  is  developed  by  industry  or  in  house,  it  must  still  be  vali¬ 
dated;  however  in-house  designs  are  easier  and  less  risky  to  validate  because  they  are 
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inherently  nonproprietary  and  easier  to  subject  to  configuration  control  meeting  the 
government’s  requirements. 

Policies  requiring  ‘early  contractor  involvement’  usually  originate  within  the 
systems  commands  either  in  written  form  or  in  an  unstated  form  reflecting  a  normal 
routine  of  business.  There  are  no  stated  policies,  directives,  or  instructions  above  the 
systems  command  level  supporting  this  position.  Indeed,  the  Navy  RDT&E  Manage¬ 
ment  Guide  (para  0624)  states,  ‘A  series  of  actions  to  contract  out  important  activi¬ 
ties,  each  wholly  justified  when  considered  on  its  own  merits,  may,  when  taken 
together,  erode  the  Government’s  ability  to  manage  its  research  and  development 
programs.*  The  decision  whether  to  involve  industry  should  be  made  on  the  basis  of 
the  need  for  services  which  are  best  provided  bv  industry,  the  policy  that  the  govern¬ 

ment  does  not  compete  with  industry,  and  the  needs  of  the  government  to  maintain 
control  of  its  development  efforts  and  to  execute  its  system  integration/design  owner¬ 

ship  responsibilities  (these  needs  do  constitute  competition  with  industiy  when  they 
dictate  in-house  development).  Where  indicated,  in-house  production  engineering 
capabilities  reside  at  Naval  Avionics  Center,  Indianapolis  (formerly  Naval  Avionics 
Facility,  NAFI)  (ref  NAFI TR-I873). 

Another  possible  transition  point  is  following  the  ED  phase  and  entering 
preproduction.  The  purpose  of  the  preproduction  phase  to  validate  the  design  data 
package  through  a  limited  or  pilot  production.  Changes  indicated  by  the  results  of 
engineering  tests  on  the  EDM  may  be  incorporated  as  well  as  changes  which  result 
from  producibility  improvements.  The  validation  or  proofing  function  is  sometimes 
overlooked  because  the  EDM  contractor  is  allowed  to  do  the  preproduction;  further¬ 
more,  the  preproduction  phase  is  sometimes  used  to  correct  gross  deficiencies  in  the 
EDM  which  totally  obscure  the  objectives  of  preproduction.  In  other  words,  pre- 
production  is  sometimes  used  to  correct  the  sins  committed  in  engineering  develop¬ 
ment  rather  than  validate  a  finished  design  package,  which  is  its  intended  purpose. 
The  nature  of  the  textbook  preproduction  phase  is  such  that,  if  a  design  data  package 
is  ever  going  to  be  used  for  competitive  procurement  or  by  a  facility  other  than  that 
producing  the  design,  it  must  bo  executed  through  unbiased  analysis  and  stringent 
change  control  procedures.  Since  the  government  is  assuming  the  responsibility  for 
the  accuracy  of  the  design  package,  it  follows  that  these  functions  should  be  per¬ 
formed  by  qualified  government  personnel.  This  may  vary  from  government 
production  personnel  acting  as  plant  representatives  for  the  program  to  execution  of 
the  entire  preproduction  phase  inhouse.  The  latter  approach  is  certainly  most  appro¬ 
priate  for  large-quantity  or  high-dollar-volume  (greater  than  $20M  in  development) 
programs,  and  it  is  the  lowest-risk  approach  to  the  preproduction  problem.  There  is 
also  a  preproduction  phase  in  functionally  controlled  procurements  which  require 
development,  but  in  this  situation  the  government  is  only  ensuring  that  the  produc¬ 
tion  contractor  can  indeed  produce  his  design  (therefore,  it  is  important  to  require 
that  the  same  facilities  be  utilized  for  preproduction  as  for  production). 

The  optimum  transition  to  industry  for  in-house  developed  designs  and  for 
designs  validated  in  house  is  following  the  preproduction  phase.  A  proofed  data  pack¬ 
age  is  available,  and  manufacturing  specialty  houses  (i.e.,  no  development  capability 
with  significantly  lower  overhead  costs)  can  be  solicited.  The  in-house  technical  team 
is  in  a  position  to  provide  strong  support  to  aid  and  monitor  the  contractor,  and  the 
government  has  configuration  control.  The  system  integration  is  complete  and  within 
the  control  of  the  government.  The  procurement  is  on  a  fixed-price  basis.  The 


technical,  cost,  and  schedule  risks  are  all  minimal.  Since  release  to  production  cannot 
take  place  until  Approval  for  Production  (see  chapter  VII)  is  issued,  there  is  ample 
time  to  incorporate  changes  dictated  by  the  NTE/OPEVAL,  to  negotiate  the  procure¬ 
ment  package,  and  to  conduct  first-article  tests.  If  the  transition  is  delayed  until  after 
NTE/  OPEVAL,  a  long  administrative  lead  time  must  be  tolerated  before  the  initial 
equipments  are  available  for  fleet  introduction;  this  transition  point  is  only  recom¬ 
mended  for  small  quantities  being  procured  from  one  source  at  a  time. 

However  the  transition  to  industry  is  performed,  the  risks  are  minimized  at 
each  step  if  the  objectives  of  the  succeeding  steps  are  made  to  influence  the  technical 
effort.  This  means  strong  Astern  engineering  control  within  the  government  and  con- 
tractural  incentives  on  industry  involvement. 


PROCUREMENT  ALTERNATIVES 


The  offices  and  agencies  of  the  government  have  evolved  a  seemingly  limitless 
variety  of  procurement  practices.  In  general,  what  has  always  been  practiced  by  a 
group  tends  to  be  what  is  practiced  on  any  procurement  by  that  group  regardless  of 
the  circumstances.  When  the  group  is  dealing  with  a  limited  scope  of  products  and 
with  the  same  set  of  circumstances,  this  blind  approach  has  generally  proved  success¬ 
ful;  however,  alterations  in  the  procurement  circumstances  usually  lead  to  cost  over¬ 
runs  and  inferior  products  unless  the  procurement  practices  are  altered  appropriately. 
This  section  deals  with  the  alternatives  which  are  available  to  tailor  the  procurement 
method  to  the  circumstances. 

The  primary  procurement  variable  is  that  which  is  to  be  procured.  Various  po¬ 
tential  suppliers  are  organized  in  a  number  of  different  ways  in  order  to  promote  the 
greatest  efficiency  and  least  risk  in  creating  their  primary  products;  the  procurement 
method  should  favor  those  suppliers  which  are  in  the  best  position  to  supply  the  prod¬ 
uct  of  interest.  The  product  consists  of  the  end  item  (component,  equipment,  system, 
etc.)  and  of  a  level  of  service.  The  services  which  may  make  up  part  of  a  product  are: 

Research 
Development 
Production 
Support  services 

The  level  of  service  required  for  a  product  is  the  most  significant  factor  in  the  deter¬ 
mination  of  a  procurement  method.  Each  service  capability  requires  special  resources; 
the  maintenance  of  those  resources  is  a  burden  on  the  company’s  cost  of  doing  busi¬ 
ness.  All  suppliers  aro  primarily  selling  production  services  and  maintain  other  serv¬ 
ices  in  support  of  those  efforts.  In  general,  the  greatest  investment  return  is  in 
providing  production  services.  On  the  other  hand,  research  generally  provides  no 
direct  return  on  the  investment;  however,  very  substantial  returns  are  generally  real¬ 
ized  when  a  research  product  becomes  successful  in  production. 

The  range  of  services  provided  differs  substantially  among  the  various  sectors 
of  suppliers.  In  the  ‘‘consumer”  sector  of  many  products,  companies  are  geared  for 
production  and  offer  extensive  support  services  through  warranties,  field  representa¬ 
tives,  and  local  repair  activities,  development  capabilities  are  generally  minimized  to 
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those  necessary  to  maintain  a  competitive  product  line.  The  “industrial”  sector  com¬ 
panies  are  similar  to  their  consumer  sector  counterparts  except  that  a  research  capa¬ 
bility  and  an  expanded  development  capability  are  maintained  to  support  high- 
technology  product  lines.  In  both  consumer  and  industrial  sector,  high-volume  pro¬ 
duction  is  the  goal  and  the  key  to  product  profitability.  The  defense  sector  is  differ¬ 
ent. 


The  defense  sector  is  characterized  by  intensive  research  and  development  and 
by  low  production  volumes,  since  the  products  are  often  military-peculiar  and  re¬ 
quired  in  small  quantities.  The  companies  participating  in  the  defense  sector  have 
well  established  R&D  capabilities,  limited  production  capabilities  which  are  set  up 
specifically  for  low-volume  production,  and  virtually  nonexistent  support  services. 
Even  companies  which  participate  heavily  in  both  industrial  and  defense  sectors  are 
usually  organized  to  segregate  the  industrial  and  defense  capabilities,  and  each  part 
conforms  to  the  characteristics  of  its  sector. 

In  recent  years,  high-technology  industrial/consumer  markets  have  seen  the 
introduction  of  many  products  which  are  readily  adaptable  to  military  applications, 
and  a  great  deal  pressure  has  been  applied  to  DoD  to  use  both  commercial  end 
items  and  services,  especially  warranty  services.  The  commercial  sector  is  not  pre¬ 
pared  for  the  extensive  documentation  and  quality  assurance  provisions  nor  for  mili¬ 
tary  standard  parts  requirements  which  are  common  to  many  DoD  procurements. 
Likewise,  the  defense  sector  is  not,  in  general,  prepared  to  deal  with  long-term  war¬ 
ranty  clauses  nor  field  service  provisions,  and  large  production  runs  will  often  have 
higher  unit  costs  than  non-DoD  sector  production  runs. 

The  alternatives  which  should  be  considered  to  make  the  most  efficient  use  of 
private  sector  talents  include  development  alternatives  which  lead  to  an  open  com¬ 
petitive  production  (as  opposed  to  closed  competition  which  is  limited  to  the  multiple 
developers);  closed-end  contracts  for  each  program  phase  (the  product  of  each  con¬ 
tract  is  sufficient  for  a  different  contractor  to  perform  the  next  phase);  split-tasking 
(each  task  is  performed  by  a  supplier  experienced  in  the  required  service);  and  the  use 
of  in-house  resources  to  plug  gaps,  as  well  as  the  traditional  single  supplier  methods. 
Neither  closed-end  contracting  nor  split-tasking  is  always  efficient  because  the  data 
requirements  to  effect  handovers  between  contractors  may  be  excessive  and  impossi¬ 
ble  to  obtain  in  the  detail  needed;  each  case  must  be  reviewed  on  its  own  merits. 
Split-tasking  will  tend  to  be  very  efficient  if  a  system  can  be  partitioned  into  equip¬ 
ments  requiring  development  and  units  which  can  be  purchased.  In  any  case,  the  pro¬ 
curement  solicitations  should  require  suppliers  to  show  thoir  management  approach 
and  their  resources  to  apply  toward  any  services  which  would  not  be  in  their  normal 
market  requirements;  this  demands  some  knowledge  of  the  various  probable  respond¬ 
ers  to  the  solicitation: 


TOTAL  CONTRACT  COST 


I - + 


BASE  PRODUCT  COST 
(100%) 


COST 

UNCERTAINTY  (5-40%) 


COMMERCIAL  SECTOR  . 

70-90%  ' 

INDUSTRIAL  SECTOR  . 

85-105%  * 

_ DEFENSE  SECTOR 

110-180% 
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The  type  of  contract  to  be  employed  is  another  procurement  variable.  A  fixed 
price  type  contract  is  most  desirable  as  the  risk  elements  are  most  firmly  established 
and  can  be  minimized.  However,  fixed  price  contracts  cannot  be  used  outside  well  es¬ 
tablished  limits  (see  chapter  XVI,  Types  of  Contracts).  In  general,  situations  allowing 
fixed  price  contracting  will  favor  commercial  supplier-type  services. 

The  selection  of  the  procurement  scheme  to  be  used  must  be  made  in  consid¬ 
eration  of  the  development  alternative  being  pursued  (see  chapter  XVII,  Development 
Alternatives).  When  the  procurement  quantity  is  low,  the  development  considerations 
will  predominate,  but  high  procurement  quantities  demand  attention  the  potential 
costs  of  each  development  procurement  combination.  Getting  industiy  involvement 
early  in  the  program  may  be  important  to  the  ultimate  cost  of  the  product.  Obviously, 
pursuing  a  development  alternative  which  transitions  the  technical  tasks  to  industry 
early  in  the  program  gets  industry  involved;  however,  only  the  limited  and  parochial 
views  of  one  company  or  a  few  (for  parallel  developments)  companies  may  be  obtained 
rather  than  an  industry-wide  view.  Another  method  uses  multiple-step  procurements 
which  obtain  industry  views  through  the  formal  procedures  of  solicitation,  proposal, 
and  evaluation-negotiation;  multiple  step  procurements  can  only  obtain  a  limited 
amount  of  precontract  industry  involvement  because  each  company  has  limited  time 
to  review,  react,  and  respond  and  because  the  legal  restrictions  can  hamper  an  open 
technical  exchange.  Under  some  circumstances,  the  various  industry  associations  can 
provide  limited  information  which  can  guide  a  development-procurement  approach; 
on  large  projects,  this  method  can  be  effective  when  it  is  supplemented  by  “independ¬ 
ent"  studies  of  industrial  capabilities,  processes,  and  limitations.  Another  approach 
which  gets  industry  involved  is  an  adaptation  of  the  commercial  airlines  acquisition 
methodology.  This  method  can  only  be  applied  under  certain  well  defined  circum¬ 
stances  including  large  in-service  quantities,  multiple  buys,  and  mature  technologies, 
but  its  advantages  include  very  low  development  dollar  involvement  and  competitive 
incentives  maintained  well  into  the  support  phase.  ARINC  Research  Publication 
1313-01-1-1447,  “Application  of  the  Commercial  Airlines  Acquisition  Methodology  to 
Department  of  the  Navy  Electronic  Equipment  Acquisitions,"  dated  15  October  1975 
(AD/A  015  694),  details  the  steps,  advantages,  and  restrictions  *.f  this  method. 

When  considering  the  procurement  alternatives,  there  are  three  points  to 
remember: 

1.  Be  a  good  customer 

2.  Remember  the  ultimate  user 

3.  Maintain  competition  as  long  as  possible 

Particularly  when  working  with  commercial  suppliers,  it  is  important  to  consider 
their  normal  modes  of  operation,  and  to  conform,  as  much  as  possible,  to  the  charac¬ 
teristics  of  their  usual  market.  Minimize  unusual  requirements  such  as  abnormally 
tight  tolerances,  heavy  data  and  documentation  requirements,  and  extensive  quality 
assurance  provisions.  It  is  desirable  always  to  work  with  the  suppliers  and  to  avoid 
adversary  circumstances.  In  working  with  industry  and  getting  industry  involvement, 
do  not  forget  the  requirements  of  the  ultimate  user.  It  may  be  necessary  to  trade  off 
some  of  the  desirable  characteristics  of  the  product  if  they  cannot  be  incorporated 
cost-effectively  —  but  do  not  compromise  any  of  the  required  characteristics.  If  con¬ 
ferences  are  held  to  obtain  industry  involvement,  invite  user  participation,  also. 
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Every  supplier  is  in  business  for  gain  and  is  entitled  to  a  fair  and  reasonable  profit; 
however,  the  government  as  the  buyer  is  entitled  to  product  with  value.  Competition 
is  the  most  effective  regulator  of  suppliers  in  assuring  a  continued  product  with 
value.  Whenever  possible,  maintain  competition  through  multiple  procurements  from 
multiple  suppliers  based  on  past  experience  of  supplier  performance;  the  details  of  fu¬ 
ture  procurement  based  on  past  performance  can  be  established  contractually.  This 
method  can  obviate  the  need  for  restrictive  specifications  used  to  weed  out  potentially 
weak  suppliers.  It  is  important,  also,  to  avoid  requirements  which  are  proprietary  to 
one  company  or  which  greatly  advantage  one  company  over  all  others;  this  is  a  par¬ 
ticularly  important  point  to  monitor  when  industiy  involvement  is  solicited  because 
one  company’s  “good  idea”  can  represent  a  proprietary  advantage.  When  it  becomes 
necessary  to  end  competition,  all  future  costs  should  be  fixed  contractually  with 
incentives  and  penalties  established  as  appropriate.  This  is  not  possible  in  total  for 
most  situations,  but  it  is  a  goal  to  be  approached.  LCC  and  DTC  procurements  are 
examples  of  contractually  fixing  future  costs. 

The  procurement  approach  is  an  effective  tool  for  reducing  costs  and  promot¬ 
ing  product  value  when  it  is  selected  to  bridge  the  conditions  of  the  product  develop¬ 
ment  with  the  circumstances  of  the  potential  suppliers.  It  should  be  selected  with 
attention  to  the  user  requirements  as  well  as  quantities,  costs,  teclmical  risk,  and 
potential  supplier  capabilities. 
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VII.  APPROVAL  FOR  PRODUCTION  (AFP) 

Deviations ...  page  VII-2 

Approval  for  Limited  Production  (AFLP) ...  VII-3 


m  APPROVAL  FOR  PRODUCTION  (AFP) 


Approval  for  Production  (AFP)  15  granted  only  after  certain  set  condi¬ 
tions  have  been  met;  AFP  must  be  granted  before  equipments  can  be 
introduced  into  the  Fleet.  Each  Systems  Command  maintains  instruc¬ 
tions  conforming  to  DoD  policy.  This  chapter  summarizes  these 
instructions.  Because  the  conditions  for  AFP  are  mandatory,  plans  to 
achieve  them  must  be  integrated  into  project  planning  early  to  ensure 
timely  approval. 

Approval  for  Production  must  be  granted  before  an  equipment  or  system  can 
be  committed  to  mqjor  production.  AFP  can  be  granted  until  tho  following  prerequi¬ 
sites  have  been  completed.  Each  of  these  items  is  reviewed  by  an  Acquisition  Review 
Board  (ARB)  which  certifies  readiness  for  production.  A  satisfactory  production  readi¬ 
ness  review  and  an  approved  Acquisition  Plan/Strategy  Paper  for  the  production  con- 
tract(s)  are  required  for  ARB  certification. 

1.  At  least  the  initial  operation  test  evaluation  is  complete  and  methods  for 
correction  have  been  confirmed.  A  Test  and  Evaluation  Master  Plan  is 
required,  and  OPTEVFOR  concurs  the  system  is  ready  for  production. 

2.  At  least  the  minimum  performance  requirements  (including  reliability  and 
maintainability  (R&M))  of  the  approved  developments  proposal  have  been 
achieved  and  a  planned  maintenance  system  has  been  developed.  The  R&M 
and  QA  programs  have  been  certified  to  comply  with  policy,  and  reliability 
design  review  actions  have  been  closed  out. 

3.  Integrated  logistic  support  planning  has  progressed  to  the  point  that  there 
is  assurance  that  all  elements  of  logistic  support  will  be  available  in 
approved  form  upon  delivery  of  the  first  production  item.  A  Logistics  Re¬ 
view  Group  (LRG)  must  review  and  certify  that  the  ILS  implementation  is 
satisfactory. 

4.  Technical  documentation  necessary  for  support  of  the  system  or  equipment 
has  beon  identified  and  technical  manuals  have  been  assured  with  first 
deployment  of  production  item. 

5.  Personnel  requirements  are  assured  for  fleet  operation  and  maintenance. 

6.  The  configuration  management  product  base  line  has  been  established  for 
identification  and  future  configuration  changes.  Specifications  and  provi¬ 
sioning  documentation  meet  the  logistics  requirements  and  are  consistent 
with  the  system  configuration. 

7.  A  system  safety  program  (MIL-STD-882)  has  been  established.  When 
explosives  are  utilized,  a  safety  review  and  recommendations  by  the  Sys¬ 
tem  Explosive  Safety  Review  Board  are  required. 

8.  The  production  phase  has  been  properly  budgeted  to  meet  requirements. 

When  the  above  prerequisites  have  been  met,  the  Systems  Commander  or 
SECNAV-designated  Project  Manager  will  prepare  the  AFP  recommendations.  These 
recommendations  are  submitted  in  accordance  with  OPNAVINST  5000.42. 
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The  final  approving  authority  depends  on  the  magnitude  of  the  project.  ACAT 
I  and  II  programs  are  approved  at  the  SECNAV  level.  ACAT  1U  programs  are  ap¬ 
proved  by  the  OPNAV  Program  Sponsor.  ACAT  IV  programs  are  approved  by  the  Sys¬ 
tems  Command  Commander  with  information  to  the  OPNAV  Program  Sponsor.  If 
OPTEVFOR  does  ?.ot  concur  in  the  recommendation,  the  Vice  CNO  resolves  the  deci¬ 
sion  for  ACAT  III  and  IV  programs. 

Once  an  equipment  or  system  is  AFP,  reapproval  will  not  be  required  unless 

1.  The  equipment  or  system  proves  deficient, 

2.  The  equipment  or  system  is  to  be  used  in  a  different  operational  environ¬ 
ment,  or 

3.  The  equipment  or  system  undergoes  a  significant  alteration.  SYSCOM 
Commanders  and  PMs  notify  the  system/equipment  OPNAV  Program 
Sponsor  if  an  AFP  file  number  is  to  be  canceled  because  the  equipment  or 
system 

a.  Has  been  modified  or  altered  to  the  extent  that  a  new  AFP  has  been 
obtained, 

b.  Is  no  longer  in  use,  or 

c.  AFP  has  been  withdrawn. 

ASN(S&L)  is  responsible  or  the  maintenance  of  records  of  AFP  actions. 


DEVIATIONS 


In  cases  of  extreme  urgency  or  military  necessity  (such  as  QRC  or  RDC),  lim¬ 
ited  production  approval  may  be  obtained  in  advance  of  AFP.  SECNAV  (ASN  (S&L)) 
must  approve  the  waiver  of  the  AFP,  and,  in  case  of  mqjor  programs,  final  approval 
must  be  obtained  from  the  SECDEF.  All  such  requests  shall  include 

1.  Quantities  to  be  produced  or  procured  and  justification  for  limited  produc¬ 
tion  in  advance  of  AFP, 

2.  Minimum  quantity  needed  to  accomplish  evaluations  on  which  to  base 
AFP, 

3.  Analysis  of  all  feasible  alternatives  waiver, 

4.  Cost,  schedulo,  and  performance  impact  of  each  alternative  on  the  pro¬ 
gram, 

5.  Results  of  T&E  conducted, 


6.  Proposed  revised  test  plan,  including  plan  for  obtaining  AFP, 

7.  US  plans,  and 

8.  Statements  of  risks,  including  alternative  courses  of  action. 


APPROVAL  FOR  LIMITED  PRODUCTION  (AFLP) 


When  sufficient  operational  testing  to  support  a  final  determination  of  AFP 
cannot  practicably  be  accomplished  prior  to  making  initial  production  commitments, 
an  AFLP  can  be  granted  for  initiating  orderly  first-lot  limited-production  runs. 

The  procedure  for  obtaining  a  AFLP  is  the  same  as  the  procedure  for  obtain¬ 
ing  an  AFP  except  that  the  requirements  on  the  prerequisites  are  less  stringent.  An 
AFLP  requires  the  following  prerequisites: 

1.  Completion  of  Test  and  Evaluation  Master  Plan  with  corrections  on  defi¬ 
ciencies  to  be  considered  and 

2.  Production  specification  requirements  and  procedures  for  confirming  that 
the  reliability  and  maintainability  will  be  achieved. 

3.  Approved  plans  for  achieving  AFP. 
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Introduction  of  equipments  into  the  Fleet  is  entirely  determined  by  the 
prior  phases.  The  planning  for  a  smooth  phase-in  of  the  new  equip¬ 
ments  and  phase-out  of  obsoleted  equipments  must  be  accomplished  at 
least  3  years  prior  to  the  actual  task  commencement.  Therefore,  the 
tasks  described  in  this'  chapter  must  be  planned  during  the  validation 
phase  or  early  in  full-scale  development. 

After  the  new  equipments  are  phased  in,  their  performance  should  be 
monitored  to  ensure  that  no  unforeseen  problems  have  been  introduced 
and  to  assure  satisfaction  of  the  operational  requirements.  Deviations 
from  expected  performance  may  require  correction  —  at  least  in  future 
acquisitions,  if  not  immediately.  The  knowledge  gained  through  the 
acquisition  cycle  should  be  retained  and  applied  to  future  acquisitions, 
not  cast  away;  often  the  greatest  savings  are  those  realized  on  future 
projects  which  can  benefit  from  the  experience  gained  by  project  per¬ 
sonnel. 


SUPPORT  REQUIREMENTS 

System  support  elements  include  everything  needed  to  operate  and  maintain 
the  system  over  its  life.  Some  of  these  elements  are; 

Operator  and  maintenance  personnel 

Repair  spare  parts 

Test  equipment  and  tools 

Technical  manuals  (both  operation  and  maintenance) 

Facilities  (depot,  IMA,  overhaul,  calibration,  etc.) 

Transportation  and  handling  equipment 

Training  courses  and  materials 

Technical  data  for  provisioning  and  reprocurement 

Technical  data  for  support  management  effectiveness  systems 

Installation  support 

These  elements  may  be  roquired  in  varying  degrees  over  the  life  of  the  system, 
which  for  support  purposes  is  phased  as  followa: 


Acquisition 

Phase-in 
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Operations 

Phase-out 

The  various  support  requirements  for  the  acquisition  phase  vary  widely  and 
usually  differ  markedly  from  those  for  the  other  phases.  The  planning  for  each  por¬ 
tion  of  the  acquisition  phase  should  be  started  with  the  first  program  planning  effort 
and  completed  prior  to  the  initiation  of  that  portion.  Since  it  is  so  integral  to  pro¬ 
gram  planning,  no  further  discussion  is  offered  here. 

The  phase-in  support  period  starts  with  the  IOC  date  and  ends  whenever  the 
full  operations  support  is  initiated.  Phase-in  support  is  normally  procured  at  the 
same  time  as  the  equipment  and  most  often  will  consist  of  contractor-supplied  train¬ 
ing  and  initial  spare  parts.  Warranty  service  and  maintenance  contracts  also  fall  into 
this  support  category.  The  planning  for  phase-in  type  support  must  be  accomplished 
during  the  preproduction  phase  for  developed  equipments  and  prior  to  the  procure¬ 
ment  solicitation  for  on-shelf  or  modified  equipments. 

Operations  phase  support  is  the  normal  service-provided  support  which  lasts 
most  of  the  service  life  of  the  equipment.  The  operations  phase  support  requirements 
are  initially  predicted  by  the  IIS  tasks  performed  during  the  acquisition  phase  and 
then  constantly  corrected  by  the  various  support  management  effectiveness  systems 
utilized  by  the  supply  system,  training  commands,  operations  commands,  systems 
commands,  field  support  activities,  etc.  The  support  system  is  set  up  for  long-term, 
continuous  operation;  therefore,  it  is  extremely  important  to  properly  plan  the  transi¬ 
tional  support  for  an  equipment.  The  planning  requires  not  only  the  accurate  execu¬ 
tion  of  the  ILS  tasks  but  also  ensuring  the  funds  and  other  resources  are  available  to 
implement  the  ILS  recommendations.  Identifying  funds  requires  budgeting  actions 
which  require  at  least  3  years  to  produce  the  allocation.  Identifying  personnel 
resources  may  require  changing  manpower  allocations,  recruitin';,  quotas,  setting  up 
training  courses,  scheduling  a  training  pipeline,  and  so  forth  —  all  of  which  may  tAe 
4  or  more  years  before  the  personnel  are  available.  Obviously,  these  are  long  lead 
times  which  may  exceed  the  time  needed  to  procure  the  equipment;  this  puts  great 
importance  on  the  phase-in  support  planning. 

All  too  often  the  planning  for  the  operations  phase  is  incomplete,  especially  in 
the  identification  of  funding  for  spare  parts  and  training.  When  the  new  system  is  put 
into  the  Fleet  without  proper  planning  or  providing  the  initial  support,  it  looks  like  a 
mammoth  stop  function  to  the  support  system;  the  support  system  “rings'  for  many 
years  trying  to  cope  with  the  perceived  discontinuity.  The  budget  cycle  ensures  that 
this  period  will  be  at  least  3  years  longer  than  the  equipment  phase-in  period;  this 
may  constitute  a  significant  portion  of  a  single  equipment’s  service  life.  The  unsup¬ 
ported  equipment  suffers  abuses  that  often  permanently  degrade  performance  and 
reliability;  atrocious  availabilities  are  realized  (sometimes  under  10%),  thus  cheating 
the  user  out  of  a  needed  capability  and  expending  resources  (manpower  and  dollars) 
better  spent  elsewhere.  This  cycle  is  commonly  caused  by  paying  for  cost  overruns  in 
the  acquisition  phase  with  funds  initially  identified  for  support  procurement.  From 
the  standpoint  of  overall  damage  to  the  service,  it  is  far  better  to  reduce  the  number 
of  equipments  procured  or  to  delay  their  introduction  until  support  is  made  available. 

When  personnel  training  is  indicated  and  supply  system  support  is  utilized 
(almost  always),  two  milestones  are  mandatory  in  the  acquisition  phase: 
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•  Training  Plans  Conference  convened  by  the  cognizant  systems  command  at 
least  4  years  prior  to  IOC  (table  VUI-R 

•  Provisioning  Conference  chaired  by  the  Ship  Paris  Control  Center  (SPCCJ 
at  least  3  years  prior  to  IOC  (table  VKI-2) 

These  conferences  reach  agreement  on  the  actions  to  be  taken  and  the  resources 
required  to  support  the  equipment.  When  the  acquisition  program  is  such  that  these 
conferences  can’t  take  place  at  the  proper  time,  they  should  be  held  as  early  as  possi¬ 
ble,  and  the  program  should  plan  and  budget  for  the  phase-in  support.  Where  feasible 
and  cost-effective,  long-term  warranty  service  provides  an  excellent  transition  vehi¬ 
cle.  The  equipment  may  mature  and  realize  reliability  growth  in  service  without 
being  a  maintenance  burden  to  the  user  organization.  Contractor  depot  services  are 
usually  implemented  after  the  warranty  period  (3-6  years,  4  typically). 

Table  VIII-I.  Training  plans  conference  prerequisites. 

(REF:  OPNAVINST  1500.8  SERIES) 


Operator  concept 
Maintenance  concept 

Skill  and  experience  level  estimates  —  especially  if  new  skills  or  high 
experience  levels  may  be  needed 

Estimate  of  manpower  requirements 

Plans  for  Fleet  introduction 

Training  requirements  estimate 

Table  VIII-2.  Provisioning  conference  prerequisites. 

(REF:  SECNAVTNST  5000.39  &  MIL-STD-1561). 

Support  concept 

Availability  requirements 

Risk  and  hazard  assessment 

Level  of  Repair/Logistics  Support  Analysis  results 

Estimated  demand  requirements 

Essential  components  listing 

Long-lead-time  item  listing 

Standard  parts/parts-peculiar  requirements 

Plans  for  Fleet  introduction 

Tentative  support  management  plan 

A  frequently  overlooked  support  requirement  is  the  technical  data  needed  to 
set  up  and  utilize  the  support  management  effectiveness  systems.  The  most  important 
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of  these  systems  is  the  Maintenance  Data  Collection  System  (MDCS)  portion  of  the 
Maintenance  and  Material  Management  (3M)  System.  MDCS  takes  maintenance 
reports  from  the  Fleet  and  computes  logistics  parameters  and  reliability,  maintain¬ 
ability,  and  availability  (RM&A)  parameters.  The  logistics  parameters  are  straightfor¬ 
ward  calculations  of  usage  data  and  parts  demand  data  used  to  isolate  and  correct 
supply  system  deficiencies.  The  RM&A  parameters  are  used  to  isolate  latent  design 
problems,  support  documentation  problems,  training  and  manning  problems,  insuffi¬ 
cient  test  equipment  allocations,  and  installation  problems.  In  order  to  effect  the  dis¬ 
crimination  needed  to  isolate  such  problems,  the  measured/computed  RM&A  values 
must  be  compared  to  “should  perform"  values.  Unfortunately,  the  required  values, 
which  are  available  during  the  acquisition  cycle,  are  frequently  not  made  available  to 
the  logistics  coordinators  responsible  for  tracking  the  equipment.  The  problems  are 
further  exacerbated  by  the  failure  of  the  acquisition  community  to  coordinate  the 
RM&A  parameters  specified  and  tested  in  the  acquisition  phase  with  those  parame¬ 
ters  which  can  be  measured  in  the  support  phase  (see  chapter  IV,  Conceptual  Phase) 
and  by  the  failure  to  require  equipment  identification  reporting  codes  to  be  name¬ 
plate  data,  to  provide  reporting  guidance  in  the  technical  manuals,  and  to  coordinate 
with  the  MDCS  system  coordinators.  The  required  actions  are  straightforward  and 
simple  when  coordinating  actions  are  initiated  prior  to  the  equipment  qualification 
phase. 


PROJECT  PHASE-OUT  PLANNING 

The  most  overlooked  support  phase  is  phase-out.  Usually,  an  equipment  is 
phased  out  because  a  replacement  equipment  is  being  phased  in.  It  seems  that  old 
mission  requirements  continue  forever,  although  they  may  be  integrated  into  new 
missions.  As  an  equipment  nears  the  end  of  its  service  life,  various  support  elements 
become  uneconomic  and  are  dropped.  Special  training  courses  are  dropped,  mainte¬ 
nance  contracts  are  terminated,  etc.  Nobody  much  cares  except  the  poor  user  who 
still  has  one  of  the  old-timers.  In  today’s  austere  budget  environment,  the  acquisition 
program  for  the  replacement  equipment  can  hardly  be  expected  to  assist  a  lame  duck. 
Most  of  the  problems  which  can  occur  during  phase-out  can  only  be  addressed  early 
in  the  acquisition  cycle  by  establishing  levels  of  repair  and  standardization  such  that 
system-peculiar  items  are  minimal  and  are  not  system  critical  and  minimal  skills  are 
required  to  operate  and  maintain  the  system.  A  maintenance  contract  should  be  con¬ 
structed  on  a  cost-per-unit  basis.  Consideration  of  the  phase-out  problems  which  may 
occur  for  an  equipment  being  replacod  by  the  current  acquisition  may  make  it  desir¬ 
able  to  alter  the  phase-in  rate  of  the  new  equipment. 

The  project  phase-out  plans  provide  for  the  transition  of  continuing  system 
tasks  into  functional  organizations  and  for  documenting  the  project  histoiy.  The 
phase-out  plan  should  incorporate  the  project  WBS  and  should  annotate  each  task  as 
completed  or  continuing  (or  recurring).  Completed  tasks  should  show  the  completion 
date  and  the  person  or  organization  responsible.  Continuing  or  recurring  tasks  should 
show  who  was  responsible  in  the  project  organization  and  the  person  or  organization 
assuming  the  responsibility.  The  coordinating  plans  or  documents  and  any  pertinent 
data  should  be  referenced,  and  storage  points  and  holders  should  be  cited.  Common 
documents  would  include  the  ILSP,  training  plans,  configuration  baseline  and  data, 
and  reprocurement  data  as  a  minimum  with  many  other  documents  supporting  as 
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required.  The  historical  section  should  include  a  listing  of  all  permanent  project  docu- 
ments,  reports,  and  data  from  beginning  to  end.  These  items  should  cover  the  design 
and  development,  testing,  acquisition,  US,  installation,  initial  field  data,  qualify  and 
workmanship,  and  actual  costs  and  schedules  compared  to  the  planned  targets.  Any 
significant  achievements  of  the  project,  any  significant  problems,  and  any  lessons 
learned  should  be  incorporated  in  a  narrative;  certainly  any  innovations  or  patents 
should  be  mentioned.  The  project’s  successes  and  failures  provide  valuable  lessons  for 
the  many  similar  projects  which  will  undoubtedly  follow  and  can  also  point  to  areas 
in  which  organizational  policies  should  be  improved. 

MONITORING  THE  PERFORMANCE  OF 
EQUIPMENTS  IN  THE  FLEET 


There  are  many  requirements  for  tracking  the  performance  of  equipments  in 
fleet  use.  Operations  planners  need  to  know  which  ships  are  capable  of  performing  a 
particular  mission,  support  planners  need  to  know  the  types  and  quantities  of  support 
needed,  and  acquisition  planners  need  to  know  how  past  acquisitions  are  performing. 
There  are  two  primary  systems  for  tracking  equipment  performance  —  the  CASREPT 
(Casualty  Report)  system  and  the  Maintenance  Data  Collection  Subsystem  (MDCS) 
portion  of  the  Maintenance  &  Material  Management  (3M)  system.  The  two  systems 
are  distinct  in  their  methods  and  purposes  of  reporting,  and  both  systems  serve  a 
multiplicity  of  purposes. 

The  CASREPT  is  a  message  report  sent  whenever  a  failure  degrades  the  abil¬ 
ity  of  a  ship  to  perform  its  missions.  Since  a  CASREPT  implies  the  ship  is  not  “ready 
and  willing”  to  take  on  any  task  assigned,  reports  are  frequently  not  made  unless  spe¬ 
cifically  required  by  the  operational  command  or  a  mission  assignment  is  jeopardized 
by  a  failure.  Vital  equipments  are  CASREPT’d  for  a  higher  percentage  of  failures 
than  semivital  equipments  and  semivital  more  than  nonvital.  No  amount  of  encour¬ 
agement  has  seemed  to  influence  commanding  officers  to  CASREPT  in  all  required 
situations  bocause  of  the  imagined  stigma  attached.  Failures  which  are  expected  to  be 
corrected  within  a  reasonable  time  (usually  24  hours)  are  not  reported.  Besides  noti¬ 
fying  operations  planners  of  the  ship’s  reduced  capabilities,  the  CASREPT  also  mobi¬ 
lizes  support  elements  to  assist  the  ship.  There  are  three  basic  CASREPT  categories 
—  CASREPT  for  parts,  CASREPT  for  outside  assistance,  and  CASREPT  to  notify  the 
chain  of  command  “my  equipment  is  down  and  I’m  working  on  it.”  The  first  two  types 
are  by  far  the  most  prevalent.  Once  a  CASREPT  is  made,  status  reports  are  made 
periodically  and  when  new  information  is  available  until  the  CASREPT  is  closed  out 
by  a  CASCOR  (casualty  corrected)  message.  The  reporting  system  is  specified  by 
NWIP-10.  The  primary  benefits  of  CASREPT  derive  from  its  timeliness  of  reporting, 
its  close  association  of  equipment  failures  to  ship  capability,  and  its  ability  to  mar¬ 
shall  timely  support. 

The  CASREPT  Master  Data  Bank  run  by  the  Fleet  Material  Support  Office 
(FMSO),  Mechanicsburg,  can  supply  historical  CASREPT  records  by  equipment  for 
the  last  3  years  containing  the  severity,  reason  for  report,  time  to  repair,  parts  usage 
data,  etc.,  for  each  CASREPT  made.  Summary  data  are  also  available.  Also,  a  Mate¬ 
rial  Condition  Index,  which  is  a  weighted  mathematical  factor  based  on  the  number 
of  CASREPTS,  severity,  and  time  to  repair  for  an  equipment  over  a  fixed  period,  is 
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useful  in  analyzing  trends  and  the  impact  of  failures  on  fleet  operations  and  can  be 
requested  from  the  data  bank.  A  few  cautionary  notes:  time  to  repair  is  measured  in 
calendar  time  rather  than  actual  active  repair  time,  parts  usage  reflects  those  parts 
thought  to  be  needed  at  the  time  of  failure  and  may  not  be  representative  of  actual 
failures,  and  the  CASREPT  system  only  receives  reports  of  significant  failures. 

CASREPT  data  can  be  requested  through  the  cognizant  system  command 
coordinating  office  or  directly  from  FMSO. 

The  MDCS  records  all  nonpreventive-maintenance  actions  on  all  equipments 
on  the  CNO’s  selected  equipment  list  or  in  the  Current  Ship’s  Maintenance  Program 
(CSMP)  and  all  deferred  maintenance  actions.  There  is  constant  pressure  on  the  Fleet 
to  report  thoroughly,  accurately,  and  in  a  timely  manner;  as  a  result,  virtually  all 
(nonpreventive)  maintenance  actions  are  reported  on  virtually  all  equipments  by  most 
ships.  A  great  wealth  of  information  is  reported  through  MDCS.  MDCS  reporting  is 
in  accordance  with  OPNAV  43-P2.  The  primary  benefits  of  MDCS  arise  from  the 
broad  range  of  detail  information  available  on  most  fleet  equipments. 

The  detailed  information  includes  grade  and  rate  of  the  reporting  technician, 
manhours  expended,  when  the  failure  was  discovered,  the  indication  of  trouble, 
whether  the  equipment  was  up,  down,  or  partially  down,  part  failure  data,  an  esti¬ 
mate  of  the  cause  of  failure,  mean  down  time  (MDT),  the  causes  of  down  time,  mean 
time  between  corrective  maintenance  actions  (MTBCM),  mean  time  to  repair 
(MTTR),  and  technician  narratives.  This  information  is  collected  on  written  formats 
from  each  ship,  key  punched  by  a  coordinating  center  (one  on  each  coast),  and 
entered  into  a  central  data  bank  run  by  the  Maintenance  Support  Office  Department 
(MSOD)  of  FMSO.  Reliability,  maintainability,  and  availability  (RM&A)  parameters 
are  derived  from  the  raw  data  by  the  Naval  Ship  Engineering  Center,  Norfolk  Divi¬ 
sion,  or  by  the  Ordnance  Maintenance  Management  Information  Center,  NWS  Con¬ 
cord,  depending  on  the  type  of  equipment;  logistics  parameters  are  derived  by  MSOD. 
MDCS  data  can  be  obtained  from  MSOD  or  through  the  cognizant  system  command 
coordinator. 

Both  systems  are  useful  in  performing  trend  analysis  on  equipment  problems. 
Maintenance  and  parts  usage  data  are  available  through  the  CASREPT  system,  but 
reliability  data  are  not  available  other  than  what  can  be  inferred  from  CASREPT  fre¬ 
quency.  Very  specific  maintenance  and  parts  usage  data  are  available  from  MDCS; 
additionally,  reliability  and  equipment  usage  data  can  be  derived  with  MDCS.  MDCS 
has  nearly  20  times  more  reports  over  any  particular  period  than  CASREPT,  but 
CASREPT8  are  extremely  timely  whereas  MDCS  requires  5  to  6  months  to  collect  all 
its  reports  for  a  particular  month.  Both  systems  are  capable;  however,  most  RM&A 
tracking  requirements  are  best  met  by  MDCS. 

Operations  planners  and  support  planners  make  extensive  use  of  the  CAS¬ 
REPT  and  MDCS;  however,  the  acquisition  community  has  traditionally  ignored  both 
Systems  except  when  forced  to  support  “get  well’  programs  under  DART.  This  situ¬ 
ation  is  unfortunate  since  most  support  problems  are  created  within  the  acquisition 
phase  and  many  problems  can  only  be  remedied  by  the  acquisition  phase.  In  truth, 
the  acquisition  community  has  demonstrated  its  lack  of  concern  for  the  equipment  or 
its  user  once  the  procurement  is  complete. 

A  number  of  programs  are  run  by  the  support  community  utilizing  MDCS  data 
to  identify  parts  availability  problems,  training  problems,  and  equipment  reliability 
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problems.  The  program  to  identify  equipment  reliability  problems  is  Project  Inter¬ 
cept.  Project  Intercept  tracks  key  equipments  and  identifies  potential  maintenance 
and  logistics  problems  by  measuring  four  RM&A  indicators  against  established  stan¬ 
dards.  The  project  includes  all  equipments  on  the  CNO  selected  equipment  list.  The 
Intercept  indicators  are  MTBCM,  MTTR,  MDT,  and  Ao.  The  MDT  parameter  is  fur¬ 
ther  divided  into  MDT  (outside  assistance),  MDT  (parts),  and  MDT  (ships  ops).  MDT 
(parts)  is  used  to  determine  the  percent  of  parts  not  on  board  when  required.  Safety- 
related  maintenance  actions  are  counted  also.  Parts  supply  problems  require  a 
response  by  NAVSUPSYSCOM;  all  other  problems  require  a  response  by  the  cogni¬ 
zant  systems  command  for  the  equipment.  The  standards  for  each  parameter  are 
established  as  a  “level*  (should  perform)  and  “limit*  (worst  acceptable).  Ao  has  a  limit 
set  by  CNO  at  75%;  percent  parts  not  on  board  has  a  limit  set  by  NAVSUP  at  35% 
(this  limit  is  consistent  with  the  rules  governing  the  ship’s  parts  allowances).  The 
cognizant  SYSCOM  establishes  levels  and  limits  for  MTOCM,  MTTR,  and  gross 
MDT.  The  system  would  work  well  if  it  were  not  for  the  serious  problems  which  un¬ 
dermine  the  project. 

The  most  obvious  problem  lies  in  the  fact  that  the  SYSCOM  participants  in 
Intercept  are  the  3M  codes  within  the  logistics  directorates.  If  a  problem  is  inter¬ 
cepted  which  is  determined  to  require  design  changes,  some  developmental  effort,  or 
other  mqjor  engineering  effort  (i.e.,  beyond  the  capability  of  designated  field  mainte¬ 
nance  activities),  the  SYSCOM  logistics  code  is  unable  to  respond  since  it  does  not 
have  cognizance  or  funding  in  these  areas.  Rather,  the  SYSCOM  acquisition  code 
should  respond;  however,  the  acquisition  code  either  responds  by  assigning  a  low  pri¬ 
ority  or  by  ignoring  the  problem.  The  SYSCOM  acquisition  code  is  reinforced  in  this 
posture  by  OPNAV  acquisition  priorities  and  by  the  lack  of  “charter  responsibilities” 
for  “support  problems.’ 

Ideally,  levels  and  limits  for  each  indicator  would  be  derived  for  the  opera¬ 
tional  requirements;  however,  the  sources  of  this  information  were  not  made  avail¬ 
able  to  the  logistics  codes  responsible  for  setting  the  standards.  Therefore,  levels  were 
established  by  measuring  a  mean  value  over  a  2-year  period,  and  limits  were  estab¬ 
lished  at  the  90%  confidence  level  in  the  “bad"  side  of  the  level  standards.  This  meth¬ 
odology  is  incapable  of  detecting  steadily  poor  performance. 

The  inconsistencies  between  the  way  RM&A  parameters  are  specified  and  the 
way  they  are  measured  require  that  the  raw  MDCS  data  be  manipulated  to  produce 
the  indicators.  The  most  serious  problem  lies  in  the  derivation  of  MTBCM,  which 
also  affects  the  calculation  of  Ao.  Since  very  few  equipments  are  time  metered,  equip¬ 
ment  operating  time  must  be  estimated  from  ship  steaming  times.  On  some  equip¬ 
ments,  this  methodology  is  no  doubt  quite  accurate;  however,  large  uncertainties 
exist  for  many  equipments.  Equipment  operating  time  is  used  with  the  observed  fail¬ 
ure  rate  to  calculate  MTBCM,  so  large  uncertainties  can  exist  in  MTBCM.  This  is  not 
quite  so  bad  as  it  seems  since  the  same  criteria  are  applied  consistently  on  a  given 
equipment,  so  long-term  trend  analysis  is  still  valuable.  However,  short-term  reports 
and  the  comparison  of  the  calculated  value  to  the  specified  value  of  reliability  are  vir¬ 
tually  meaningless  for  some  equipments.  Furthermore,  MTBCM  is  used  to  calculate 
Ao  rather  than  MTBF.  This  is  because  MDCS  cannot  distinguish  between  a  noncriti- 
cal  failure  and  a  “relevant*  failure.  The  mechanism  is  available  in  the  MDCS  to  make 
this  distinction  through  the  “as  discovered  status  code’;  however,  no  guidance  is  pro¬ 
vided  to  the  reporting  technicians  in  determining  what  constitutes  a  “partial  failure* 
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and  what  constitutes  a  “complete  failure”  in  a  sense  that  will  provide  reporting  con¬ 
sistency  and  a  meaningful  dikinction  between  MTBCM  and  MTBF  for  a  given  equip¬ 
ment.  If  time  meters  were  provided  for  equipments  and  if  each  equipment  technical 
manual  defined  partial  and  complete  failures,  these  mqjor  problems  in  tracking  reli¬ 
ability  could  be  overcome  sufficiently  to  produce  results  with  a  usable  confidence 
level. 


Reporting  accuracy  is  a  most  important  issue  in  the  MDCS.  An  extreme 
amount  of  pressure  has  bkn  applied  by  operational  commanders  through  inspections 
to  cause  heightened  command  attention  to  3M.  Nevertheless,  the  conditions  under 
which  reports  are  produced  sometimes  are  not  conducive  to  reporting  accuracy.  This 
accuracy  can  be  improved  if  the  equipment  reporting  codes  (EIC,  APL,  serial  number) 
are  part  of  the  nameplate  information  and  if  the  equipment  technical  manuals  define 
areas  which  otherwise  cause  confusion  to  the  technician  (such  as  the  distinction 
between  a  partial  and  a  complete  failure). 

The  selected  equipment  list  is  promulgated  by  CNO  with  the  intention  of 
reducing  reporting  requirements.  While  this  is  a  tenuous  premise  for  many  ships, 
care  must  be  taken  to  avoid  not  tracking  significant  problems  and  potential  problems. 
The  present  method  seems  to  be  based  on  an  arbitrary  accept/reject  decision  made  for 
each  equipment  nominated  by  the  SYSCO  Ms.  TELCAM  recommends  that  the  selected 
equipment  list  include  all  vital  and  semivital  equipments,  all  new  equipments  for  at 
least  4  years  after  IOC  and  until  the  equipment  performance  consistently  tracks  at 
acceptable  levels  based  on,  the  operational  requirements,  and  all  warranted  items. 

The  use  of  MDCS  data  to  support  warranties  is  a  natural  application  of  the 
system.  However,  the  problem  areas  discussed  above  must  be  resolved  by  prior  plan¬ 
ning  for  an  equipment  before  MDCS  is  a  valuable  warranty  tool.  An  additional  fea¬ 
ture  to  the  system  is  to  structure  a  special  “deferred  action  taken’  code  —  say  “KW" 
—  for  warranty  reporting  and  to  put  this  code  on  the  warranty  notice  on  the  equip¬ 
ment. 


If  information  is  desired  from  both  the  CASREPT  and  MDCS  data  banks,  it  is 
best  to  forward  the  request  to  the  cognizant  system  command  coordinator  for  that 
equipment.  If  the  information  is  specifically  to  support  a  SYSCOM-sponsored  project, 
the  request  should  be  forwarded  through  the  sponsor.  At  NOSC,  the  Design  Engi¬ 
neering  Division  assists  in  formulating  information  requests,  particularly  those 
directly  to  MSOD  or  FMSO. 


UTILIZING  CONTRACTOR  SUPPORT 


There  are  two  levels  of  contractor  support  —  interim  and  long-term.  Interim 
support  is  frequently  used  during  the  phase-in  period  as  a  stop-gap  while  operations 
phase  support  is  being  set  up.  Long-term  support  is  intended  for  the  entire  service  life 
of  the  equipment. 

A  contractor  may  not  be  particularly  well  prepared  to  provide  all  the  support 
elements  which  may  be  required,  especially  some  forms  of  training,  depot-type  main¬ 
tenance,  and  test  equipment,  requiring  these  elements  of  a  contractor  will  cost  much 
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more  than  if  the  contractor  is  set  up  to  provide  them.  However,  the  supplier  of  the 
equipment  is  always  the  best  source  of  interim  support;  the  relative  inefficiencies  of  a 
contractor  in  some  services  can  usually  be  tolerated  for  the  extent  of  a  phase-in 
period.  Although  the  contractor  may  not  desire  to  be  held  responsible  for  interim  sup¬ 
port,  there  is  actually  little  choice  in  most  cases.  Contractors  not  familiar  with 
warranty  clauses  will  tend  to  balk  at  their  application,  for  instance.  The  contractual 
clauses  requiring  interim  support  should  be  clear  and  concise.  The  division  of  respon¬ 
sibility  between  the  government  and  the  contractor  should  be  clearly  defined,  and  the 
length  of  the  contractor’s  responsibilities  should  be  fixed.  Complex  contractual 
requirements  will  tend  to  undermine  the  contractor’s  ability  to  plan,  estimate,  and 
execute  his  support  responsibilities. 

Long-term  support  brings  with  it  potentially  great  benefits  and  equally  great 
pitfalls.  It  is  almost  always  economically  advantageous  to  make  use  of  a  contractor’s 
commercial  distribution  and  maintenance  system,  if  one  exists.  There  are  innumer¬ 
able  hidden  costs  in  setting  up  any  support  system,  so  when  one  already  exists,  it  is 
virtually  assured  that  it  will  be  less  costly.  Equipment  which  has  been  developed  for 
military  use  should  be  supportable  by  the  government's  facilities  if  the  equipment  has 
been  properly  designed.  Equipments  which  have  been  developed  for  a  commercial 
market  may  not  be  at  all  adaptable  to  the  government’s  system;  therefore,  the  com¬ 
mercial  support  system  is  most  appropriate,  Commercial  support  is  not  without  its 
problems;  the  equipment  operational  requirement  must  bo  reviewed  to  ensure  that 
the  existence  of  commercial  support  pitfalls  is  tolerable.  Some  of  these  pitfalls  are: 

•  Poorly  implemented  commercial  support 

•  Labor  strikes 

•  Financial  failure  of  the  company 

•  Susceptibility  to  disruption  during  conflict 

A  poorly  implemented  support  system  should  be  discovered  during  a  screen  of  candi¬ 
date  equipments  (see  chapter  XI).  Customer  comments  will  generally  reflect  the  com¬ 
petence  of  equipment  support.  A  system  may  be  poorly  implemented  for  government 
purposes  even  though  it  is  excellent  for  a  particular  market.  For  instance,  a  market 
confined  to  the  Great  Lakes  region  might  be  serviced  quite  well  from  Cleveland,  but 
the  service  would  be  deplorable  for  Diego  Garcia.  An  economic  analysis  of  the  pipe¬ 
line  assets  required  for  adequate  support  may  be  necessary  to  resolve  this  problem. 
Obviously,  some  requirements,  especially  vital  systems,  do  not  lend  themselves  to 
nonorganic  support,  much  less  contractor  support.  However,  many  requirements 
could  tolerate  the  relatively  short  delays  that  some  commercial  worldwide  support 
systems  can  provide.  These  systems  may  remain  intact  even  during  a  mqjor  conflict, 
but  use  of  their  support  services  may  be  disrupted.  An  enemy  would  not  call  a  truce 
to  allow  you  to  lake  your  broken  gidget  to  the  authorized  factory  service  center  in  his 
occupied  territory,  nor  could  you  depend  on  a  service  center  run  by  belligerent  parti¬ 
sans.  These  factors  must  be  taken  into  account  in  the  support  planning;  generally, 
these  problems  should  be  resolved  by  accepting  or  rejecting  a  candidate  equipment  on 
its  supportability  criteria. 

Any  support  system  could  conceivably  be  affected  by  a  labor  strike,  although 
government  facilities  should  be  considerably  less  susceptible.  Strike  tolerability  is  a 
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function  of  the  criticality  of  the  equipment.  Nonvital  equipments  can  tolerate  long 
shutdowns  of  some  support  elements  while  vital  equipments  cannot.  Several  options 
are  available  to  diminish  the  impact  of  a  strike: 

1.  An  injunction  can  be  imposed,  for  reasons  of  national  security. 

2.  Additional  jipeline  assets  can  be  required. 

3.  A  secondary  service  agreement  can  be  employed. 

4.  The  government  can  take  over  support  of  the  item. 

An  injunction  can  be  employed  only  in  rare  cases  during  peacetime  because  of 
its  political  effects;  it  is  primarily  a  wartime  tool.  Additional  pipeline  assets  can  nor¬ 
mally  be  made  available  only  by  very-high-volume  industries,  since  the  assets  come 
out  of  inventory.  Low-volume  industries  which  are  subject  to  frequent  labor  problems 
may  warrant  a  greater  initial  purchase  in  order  to  provide  reserve  pipeline  assets; 
this  is  subject  to  the  scrutiny  of  an  economic  analysis.  Secondary  service  agreements 
are  frequently  employed  in  commercial  practice;  in  this  case,  a  second  company  takes 
over  the  support  load  when  the  primary  system  cannot  meet  its  demands.  The  exis¬ 
tence  of  such  agreements  should  be  investigated  during  the  screening  process;  if  none 
exists,  a  requirement  that  much  an  agreement  be  created  might  be  negotiated  into 
the  contract  provided  that  the  government’s  business  is  sufficiently  large  to  contain 
the  necessary  economic  incentives.  Highly  proprietary  items  usually  cannot  employ 
blanket  secondary  service  agreements  independent  of  the  parent  company;  however, 
systems  of  independent  factory-authorized  service  representatives  are  very  effective. 
Instances  in  which  the  government  might  take  over  support  of  its  items  are  limited 
because  of  the  investment  required.  A  notable  illustration  of  such  a  case  involved  the 
overhaul  of  Polaris  guidance  sections,  which  was  originally  performed  by  the  contrac¬ 
tor.  An  18-month  striko  threatened  to  cripple  this  vital  system,  so  the  government 
took  over  the  overhaul  responsibilities.  There  are  a  number  of  arrangements  whereby 
the  government  becomes  a  secondary  service  agent  for  a  company.  Normally,  only  a 
traction  of  the  government’s  support  requirements  are  met  by  government  facilities, 
but  some  protection  is  provided  at  the  expense  of  the  cost  of  maintaining  the  capabil¬ 
ity.  Some  savings  are  still  realized  over  full  government  support. 

The  financial  collapse  of  the  contractor  could  also  destroy  the  support  system 
for  his  equipment.  Secondary  service  agreements  are  the  most  effective  protection 
against  this  problem.  However,  equipments  containing  sole  source  parts-peculiar  can 
defy  virtually  any  protective  ploy.  The  government  can  require  the  posting  of  a  surety 
bond,  but  this  only  provides  monetary  compensation  for  loss.  Meanwhile,  the  support 
of  the  equipment  is  still  nonexistent.  The  government  can  also  require  that  all  prod¬ 
uct  and  process  documentation  be  turned  over  in  event  of  contractor  failure.  In  the¬ 
ory,  the  government  can  then  set  up  its  own  support;  in  practice,  the  documentation 
is  seldom  adequate  to  allow  support  to  be  reestablished.  In  the  final  analysis,  the  only 
solution  proved  practical  is  the  second  service  agreement,  short  of  government  rescue 
of  the  company. 
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18  8  Pr°Wein  with  second  service  agreements  between  a  contractor 
and  the  government  if  an  equipment  contains  proprietary  pacts  or  designs  The  s mv- 
ernment  can  become  liable  to  damages  if  it  discloses  the  proprietaty  intonation^ 
Even  when  the  government  is  faultless,  it  may  still  become  embroiled  in  protracted 
le^l  proceedings.  In  such  cases,  the  government  has  been  found  at  fault  for  not  tak¬ 
ing  adequate  measures  to  protect  the  information. 
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IX.  LIFE-CYCLE  COSTING  (LCC) 


In  system  development  programs,  cost  is  a  mfyor  consideration  in  decisions 
involving  selection  of  candidate  systems  or  design  alternatives  Therefore,  the  cost 
concept  and  the  methodology  of  cost  analysis  to  be  chosen  and  used  in  support  of 
such  decision  making  are  of  basic  importance.  In  the  past,  cost  considerations  in  sys 
tern  acquisition  were  generally  narrow  in  perspective,  with  emphasis  being  limited 
primarily  to  the  area  of  hardware  development  and  procurement,  while  deployment 
costs  associated  with  system  operation  and  maintenance  and  support  were  given  little 
attention  or  completely  ignored.  This  traditional  cost  approach  often  resulted  in 
equipments  that  were  costly  to  maintain  and  support. 

An  alternative  approach  is  to  consider  system  life-cycle  costs  as  a  basis  for 
decision  making.  Based  on  the  total-cost  concept,  the  life-cycle  costing  technique  can 
be  a  useful  analysis  tool  and  a  decision-making  aid  throughout  the  system  develop¬ 
ment  process.  As  a  basic  advantage  of  using  this  technique,  visibility  is  ensured  for 
all  nuqor  components  of  the  system  total  cost  This  permits  decisions  to  be  made  on 
the  basis  of  the  complete  system,  and  allows  attention  to  be  focused  on  those  system 
parameters  and  support  factors  with  influence  on  system  operating  costs  as  well  ns 
hardware  acquisition  costs  Hov  sver,  implementation  of  life-cycle  costing  implies  cer¬ 
tain  requirements,  such  as  basic  cost  data  and  support  expertise.  In  addition,  the 
decision  maker  or  project  manager  must  be  familiar  with  the  underlying  concept  and 
methodology  involved. 

The  basic  elements  and  applications  of  life-cycle  costing  are  addressed  in  this 
chapter,  the  purpose  of  which  is  to  provide  the  project  manager  with  an  understand¬ 
ing  of  life-cycle  costing  and  its  applications  in  system  development. 


SYSTEM  LIFE-CYCLE  COSTS 


System  total  life-cycle  cost  is  composed  of  many  elements  representing  the 
various  resources  which  are  required  for  system  acquisition  and  operation  through¬ 
out  the  entire  life  of  the  system.  In  the  order  of  their  occurrence,  the  different  system 
cost  elements  may  be  separated  into  the  following  categories 

•  Research  and  development.  Costs  primarily  weiated  w  ith  the  develop¬ 
ment  of  a  new  system  or  capacity  to  the  point  at  which  it  is  ready  for  pro¬ 
curement  and  operational  use 

•  Investment  Costs  beyond  the  development  phase  to  introduce  a  new  sys¬ 
tem  or  capability  into  operational  use,  including  installation  and  checkout 

•  Operation  and  support  (OftS).  Recurring  costs  of  operation,  maintenance, 
and  logistics  support  of  the  system 
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The  purpose  of  identifying  and  displaying  system  costs  as  separate  categories 
is  to  facilitate  their  evaluation  by  the  decision  maker  or  planner,  since  the  costs  asso¬ 
ciated  with  each  phase  of  the  system  life  cycle  bear  a  different  relationship  to  time 
and  to  the  number  of  units  of  the  system  to  be  procured.  R&D  costs  are  relatively 
independent  of  both  time  and  the  number  of  system  units  to  be  procured.  Investment 
costs  are  also  independent  of  time,  but  are  directly  related  to  the  number  of  system 
units  to  be  deployed  and  to  the  expected  service  life  of  the  system.  Because  of  their 
relative  independence  of  time,  R&D  and  investment  costs  are  considered  as  one-time 
costs.  By  contrast,  recurrent  O&S  costs  are  long-term  costs  which,  for  systems  with 
long  service  lives,  may  account  for  the  mqjor  part  of  the  system  total  life-cycle  cost. 

COST  ESTIMATING  METHODS 

Besides  identification  of  system  cost  elements,  the  mqjor  effort  of  life-cycle 
costing  is  the  development  of  cost  estimates.  The  means  for  making  cost  estimates 
are  many  and  varied,  ranging  from  the  use  of  expert  opinions  and  informed  judgment 
as  the  only  basis  for  an  estimate  to  the  application  of  more  formal  methods.  The  more 
formal  methods  used  are  generally  of  three  types  —  statistical,  engineering,  and 
accounting. 

The  statistical  method  of  cost  estimating  is  based  on  the  use  of  historical  cost 
data  of  past  or  existing  systems  and  the  application  of  selected  statistical  techniques, 
such  as  multiple  regression  and  correlation  analysis  and  scatter  diagrams.  By  use  of 
the  statistical  techniques  the  historical  data  are  analyzed  to  find  functional  relation¬ 
ships  between  system  costs  and  specific  elements  of  system  description,  such  as 
weight,  speed,  activity  rates,  and  number  of  personnel.  The  derived  relationships, 
often  referred  to  as  cost  estimating  relationships  (CERs),  are  used  to  estimate  or  pro¬ 
ject  costs  for  new  systems  It  is  assumed  that  the  historical  data  used  are  valid  if  the 
systems  which  generated  such  data  are  sufficiently  similar  to  the  new  systems  under 
consideration.  This  type  of  cost  estimating  method  is  usual'y  applied  at  relatively 
high  levels  of  aggregation  for  cost  studies  of  advanced  systems,  particularly  during 
the  early  development  stages  of  the  system 

The  essence  of  the  engineering  method  of  cost  estimating  is  to  successively 
break  down  the  system  or  item  of  hardware  into  lower  level  components  until  mean¬ 
ingful  conjectures  about  the  cost  implications  of  the  various  components  can  be  made. 
These  estimates  are  usually  based  upon  past  experience,  analogies,  rules-of-thumb, 
and  expert  opiniou.  CERs  are  often  applied  at  this  lower  level  of  detail.  The  resulting 
estimates  for  the  individual  components,  together  with  the  cost  estimates  for  inte¬ 
grating  the  components,  are  then  combined  to  obtain  the  total  estimate.  One  of  the 
useful  features  of  the  engineering  method  is  that  it  helps  to  separate  those  parts  of 
the  system  or  problem  which  require  novel  treatment  from  those  which  can  be  dealt 
with  by  conventional  means.  However,  a  disadvantage  of  this  method  is  that  it  fro- 
que..tly  leads  to  underestimating  because  inadequate  allowance  is  made  for  the  cost 
of  integration.  Therefore,  when  the  engineering  method  is  used,  the  statistical 
method  is  often  also  applied  at  a  higher  level  of  aggregation  to  ensure  against  under¬ 
estimating.  Chapter  X1U  includes  a  method  of  improving  engineering  cost  estimates. 

The  accounting  method  relies  on  the  fact  that  certain  factors  or  estimating 
relationships  are  inherent  in  the  books  of  account,  financially  or  otherwise.  Overhead 
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rates,  labor  rates,  and  material  consumption  rates  are  examples.  The  method  is  con¬ 
ceptually  simple,  but  it  does  usually  require  that  estimates  be  made  at  a  relatively 
lower  level  of  detail  than  is  generally  practical.  Furthermore,  when  the  accounting 
method  is  used,  it  is  necessary  to  exercise  extreme  caution  so  that  misleading  impres¬ 
sions  arising  from  using  the  relationship  out  of  context  are  not  conveyed. 

The  different  cost  estimating  methods  may  be  applied  singly  or  in  combina¬ 
tion,  depending  on  the  given  problem.  Selecting  the  method  or  methods  to  be  used  is 
dependent  on  many  factors,  including  time  available  to  make  the  estimate;  level  of 
detail  and  preciseness  needed;  type  of  analysis  to  be  performed;  availability  of  system 
descriptive  information;  form  and  availability  of  relevant  historical  data;  and  extent 
of  departure  from  experience  of  the  system  for  which  cost  estimates  are  to  be 
obtained.  Ultimately,  the  final  choice  is  dependent  on  the  experience  and  preference 
of  the  individual  cost  analyst. 

SYSTEM  COST  ANALYSIS 

There  are  two  types  of  system  cost  analysis  which  are  particularly  applicable 
to  decision  making  in  system  development  —  “intrasystem"  and  “intersystem.”  The 
first  involves  the  comparison  of  different  designs  for  a  given  system,  while  the  second 
involves  the  comparison  of  a  set  of  competing  candidate  systems. 

In  ‘intrasvstem’  analysis  the  emphasis  is  on  the  selection  of  the  configuration 
or  characteristics  of  a  system.  Costs  are  used  to  assist  in  the  selection.  System  char¬ 
acteristics  are  sought  which  provide  the  minimum  cost  for  various  performance  levels 
or,  conversely,  costs  are  used  to  indicate  those  achievable  characteristics  which  maxi¬ 
mize  performance  for  various  possible  funding  levels.  This  type  of  analysis  can  be  use¬ 
ful  in  evaluating  the  cost  impacts  of  alternative  design  specifications  and  system 
operating  characteristics.  It  provides  an  in-depth  consideration  of  a  single  system, 
anA  may  be  used  to  help  put  together  an  initial  system  description. 

In  “intersvstem"  analysis,  the  emphasis  is  on  comparing  two  or  more  systems 
competing  for  the  same  mission.  Effort  is  concentrated  on  isolating  those  features  of 
each  of  the  competing  systems  that  cause  costs  to  differ.  It  is  assumed  that  each  of 
the  competing  systems  has  been  optimized,  or  suboptimized,  as  to  configuration  so 
that  the  comparison  is  meaningful,  or  “fair.”  This  type  of  analysis  is  applicable  for 
selecting  the  preferred  system,  and  is  often  applied  during  the  conceptual  phase  of  the 
system  life  cycle. 

RELIABILITY,  MAINTAINABILITY,  AND  SUPPORT  FACTORS 


Traditionally,  consideration  of  reliability  and  maintainability  during  system 
development  is  concerned  primarily  with  meeting  operational  requirements  rather 
than  reducing  system  support  costs.  As  a  result,  there  is  often  a  tendency  to  specify 
minimum  levels  of  reliability  and  maintainability  that  will  satisfy  the  system  avail¬ 
ability  requirement  without  considering  the  effects  on  the  life-cycle  cost  of  system 
support.  On  the  other  hand,  it  has  been  generally  indicated  by  results  of  studies  on 
past  and  existing  systems  that  higher  levels  of  reliability  and  maintainability  are 
warranted  in  order  to  reduce  the  high  cost  of  system  maintenance  and  logistic  sup- 
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nort  which,  for  systems  with  long  service  lifa  spans,  is  the  predominant  component  of 
system  total  life-cycle  cost. 

A  list  of  elements  associated  with  the  cost  of  system  support  is  shown  in  figure 
IX-1,  together  with  the  principal  factors  which  contribute  to  mission  readiness.  Note 
that  the  support  elements  are  not  independent  of  each  other,  but  are  defined  by  the 
total  support  plan.  Each  element  must  be  properly  related  with  every  other  element 
in  order  to  provide  adequate  maintenance  and  logistic  support  for  the  system  equip¬ 
ments  over  the  life  cycle  of  the  system.  If  the  definition  or  scope  of  one  of  the  ele¬ 
ments  is  altered,  changes  are  induced  in  others  which  may  significantly  affect  the 
coherence  of  the  support  system  and  cause  redundancy,  mismatch,  or  omission  of  vi¬ 
tal  factors. 


Figure  IX-1.  Sysiem  support  in  life-cycle  cost 


Such  changes  may  have  impact  not  only  upon  the  effectiveness  of  system  support  but 
also  upon  the  total  life-cycle  cost  of  the  system 


The  effects  of  reliability  and  maintainability  on  support  cost  can  be  seen  by 
considering  the  largest  item  in  the  support  cost  package  —  the  cost  of  maintenance, 
which  is  the  combined  cost  of  maintenance  personnel  and  spare  parts  used  for  repair. 
The  number  of  spare  parts  used  for  repair  is  directly  related  to  the  number  of  equip¬ 
ment  failures  which  occur  during  a  given  period,  while  the  workhours  needed  for  cor¬ 
rective  maintenance  is  related  to  both  the  number  of  failures  during  the  period  and 
the  mean  time  to  repair  (MTTR),  which  is  a  measure  of  maintainability.  Since  the 
number  of  equipment  failures  occurring  during  any  time  period  is  a  function  of  equip¬ 
ment  reliability,  or  the  mean  time  between  failures  (MTBF),  it  follows  that  the  costs 
of  maintenance  personnel  and  spare  parts  will  be  high  for  low  reliability  and  main 
tainability.  Furthermore,  a  low-reliability  system  will  also,  over  the  life  cycle,  require 
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more  test  equipment,  technical  manuals,  and  other  support  items  such  as  facilities 
and  spare  parts  transportation. 

Thus,  reliability  affects  all  support  cost  elements,  directly  or  indirectly.  Main¬ 
tainability  is  important  for  similar  reasons.  In  general,  it  may  be  shown  that  the  sys¬ 
tem  acquisition  costs  (R&D  and  investment)  increase  with  reliability,  while  the  recur¬ 
rent  costs  of  operation  and  support  decrease.  These  relationships  are  shown  in  figure 
IX-2. 


Figure  IX-2.  Life-cycle  cost  vs  reliability  (Relationships  for  maintainability  are  similar.) 


Maintenance  and  support  planning  is  another  area  with  important  influence 
on  system  total  life-cycle  cost.  Its  function  is  to  provide  a  total  support  program  for 
the  system  being  designed  and  developed.  Ideally,  the  total  support  program  should 
be  well  balanced  and  tailored  for  the  system  under  development  in  the  interest  of  effi¬ 
ciency  and  economy.  However,  the  impact  of  maintenance  and  support  planning  on 
support  cost  is  felt  far  upstream  from  equipment  delivery.  Therefore,  the  maintenance 
and  support  effort  should  be  started  early  in  the  development  program  if  it  is  to  have 
an  opportunity  to  explore  the  various  concepts  for  supporting  the  system  and  to  influ¬ 
ence  design  decisions  in  system  maintenance  and  support  features.  System  support 
cost  must  also  be  emphasized  early  and  must  be  included  as  a  basis  for  evaluating  and 
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selecting  the  maintenance  and  support  alternatives  during  the  various  planning 
stages,  while  cost  analysis  and  tradeoff  (such  as  level  of  repair  analysis)  should  be 
performed  to  help  establish  the  repair  strategies  and  logistics  support  requirements. 
As  the  system  development  progresses  and  information  necessary  for  spelling  out  the 
support  program  becomes  available,  the  maintenance  and  support  plans  should  be 
appropriately  defined  at  each  stage.  This  will  provide  a  base  from  which  life-cycle  cost 
of  system  support  can  be  evaluated  for  use  in  program  decisions. 

SYSTEM  DEVELOPMENT 


System  development  is  a  process  involving  a  series  of  engineering  activities 
and  management  decisions  the  aim  of  which  is  the  successful  selection  and  specifica¬ 
tion  of  a  preferred  system  design  for  procurement  and  production.  System  life-cycle 
costs  must  be  a  part  of  the  basis  of  decision  making  involving  the  selection  of  system 
candidates  or  design  alternatives.  The  decision  maker  must  take  into  account  the 
future  cost  implications  of  system  operation  and  support  as  well  as  the  near-term 
costs  of  hardware  procurement  and  installation.  The  primary  purpose  of  life-cycle 
costing  is  to  help  define  and  describe  these  cost  implications. 

CONCEPTUAL  PHASE 

The  conceptual  phase  consists  of  many  activities,  including  identification  and 
definition  of  conceptual  systems;  technical  feasibility  and  tradeoff  studies,  and  experi¬ 
mentation  and  test  of  operational  requirements,  key  components,  and  critical  subsys¬ 
tems.  The  output  of  this  phase  includes  a  set  of  alternative  technical  approaches  or 
candidate  systems.  As  indicated  in  figure  IX-3,  the  role  of  life-cycle  costing  during  the 
conceptual  phase  is  to  assist  in  the  selection  of  the  preferred  system.  Depending  on 
the  type  of  system  development  involved,  a  candidate  system  may  be  evaluated 
according  to  preestablished  criteria  (such  as  design-to-cost  goals),  or  its  cost  perform¬ 
ance  may  be  compared  with  that  of  an  existing  system  that  is  to  be  replaced.  Since 
Systems  in  the  early  stages  of  the  system  life  cycle  are  described  in  terms  of  broad 
performance  parameters  and  concepts,  cost  estimates  of  the  candidate  systems  wdl  be 
provided  at  relatively  high  levels  of  aggregation.  Such  cost  estimates  are  usually 
obtained  from  empirically  derived  cost  estimating  relationships,  that  is,  by  statistical 
methods.  The  main  purpose  of  the  estimates  during  this  phase  is  to  detect  cost  differ¬ 
ences  among  the  candidate  systems.  The  emphasis  is  on  indicating  the  cost  impacts  of 
the  mqjor  system  features  rather  than  on  the  detail  elements  of  the  individual  sys¬ 
tems. 


The  type  of  analysis  used  for  comparing  the  candidate  systems  and  their  costs 
depends  on  whether  other  factors  besides  cost,  such  as  system  effectiveness  and  per¬ 
formance,  are  also  included  as  criteria  for  the  preferred  system  selection.  Some  form 
of  cost-effectiveness  analysis  will  be  indicated,  for  example,  if  system  effectiveness  is 
a  consideration  and  can  be  meaningfully  quantified.  For  cases  in  which  emphasis  is 
on  minimizing  cost  rather  than  maximizing  performance  (or  effectiveness),  a  method 
for  comparing  the  candidate  systems  and  selecting  the  preferred  system  is  simply  to 
fix  or  specify  the  level  of  system  performance  required  and  identify  the  system  with 
the  lowest  cost  among  the  alternatives.  The  system  identified  is  the  minimum-cost 
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Figure  IX-3.  Ufe-cycle  cosling  in  system  development 
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solution  for  the  specified  level  of  performance.  It  is  assumed,  in  this  type  of  system 
comparison,  that  any  extra  performance  possessed  by  an  individual  candidate  above 
the  level  required  is  of  little  value  and,  therefore,  does  not  warrant  extra  cost. 

Since  the  stress  is  on  comparing  system  costs,  application  of  life-cycle  costing 
■  during  the  conceptual  phase  depends  on  whether  alternatives  are  available.  In  addi- 

i  tion,  since  cost  estimates  during  the  early  stages  of  the  system  life  cycle  are  usually 

coarse,  application  of  life-cycle  costing  during  the  conceptual  phase  is  most  meaning- 
I  ful  when  there  are  significant  differences  among  alternative  systems.  In  any  event, 

'  life-cycle  costs  should  be  considered  as  early  as  possible  for  maximum  influence  on 

1  the  system  and  the  associated  costs  of  operation  and  support. 

VALIDATION  PHASE 


The  validation  phase  begins  with  system  requirements  established  in  the  con¬ 
ceptual  phase  (fig  IX-1),  examines  different  design  concepts  and  approaches,  and 
identifies  a  cost-effective  system  design  for  the  following  engineering  effort.  The  vali¬ 
dation  phase  is  also  concerned  with  system  maintenance  and  support  concept 
alternatives  as  well  as  system  operating  modes.  The  outputs  include  the  selected  sys¬ 
tem  design  specifications  (development  specification)  and  the  technical  and  program 
plans  for  the  next  phase. 


Specifically,  life-cycle  costing  may  be  applied  during  the  validation  phase  to 
assist  in  the  following 


Evaluation  an?  selection  of  system  designs 


•  Design  analysis  and  tradeoff 

•  Evaluation  and  selection  of  maintenance  and  support  concepts 

•  Evaluation  of  cost  impacts  of  parameter  changes  (such  as  reliability  and 
maintainability) 


•  Assessing  cost  implications  of  equipment  selection  options  (commercial  vs 
military  equipment,  build  vs  buy  or  modify,  etc.) 


Application  of  life-cycle  costing  during  validation  is  also  useful  in  evaluating 
and  selecting  maintenance  and  support  concepts.  This  is  particularly  important  since 
this  is  the  stage  at  which  system  maintenance  and  support  concepts  are  initially 
defined  and  considered.  Note,  however,  that  as  more  detailed  elements  of  the  system 
are  being  considered,  the  life-cycle  costing  efiort  required  during  the  validation  phase 
is  expected  to  be  much  greater  than  that  required  for  the  previous  phase.  Therefore  a 
life-cycle  costing  model  is  generally  necessary  in  order  to  reduce  the  cost  estimating 
burden  as  well  as  to  ensure  the  availability  of  timely  results. 


FULL-SCALE  DEVELOPMENT 


The  full-scale  development  phase  is  the  intensive  engineering  phase  during 
which  the  system,  including  all  the  items  necessary  for  its  support,  is  designed. 
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fabricated,  and  tested.  The  output  is  a  model  of  the  system  configuration  which  is 
demonstrated  and  evaluated  to  meet  all  requirements  based  on  the  specifications  gen¬ 
erated  during  validation.  The  chosen  system  design  concept  is  examined  in  detail,  and 
changes  or  modifications  necessary  to  complete  and  optimize  the  system  configura¬ 
tion  are  introduced.  The  original  design  concept  may  even  be  replaced,  if  its  pursuit 
proves  to  be  undesirable  or  if  it  is  found  to  have  serious  shortcomings  as  it  is  refined 
during  the  engineering  stage.  The  role  of  life-cycle  costing  during  this  phase  is  pri¬ 
marily  to  help  evaluate  such  major  design  changes  by  assessing  their  cost  impacts, 
particularly  in  the  areas  of  maintenance  and  support.  Life-cycle  costing  may  also  be 
input  to  decisions  such  as  the  choice  of  standard  or  nonstandard  parts.  Since  engi¬ 
neering  design  entails  many  changes,  it  is  apparent  that  there  are  many  areas  involv¬ 
ing  a  choice  among  alternatives  where  cost  is  a  consideration.  However,  the  emphasis 
in  life-cycle  costing  must  be  on  major  items  or  areas,  as  it  is  not  practical  to  optimize 
eveiy  detail  element  of  the  system. 

ADDITIONAL  AREAS  OF  APPLICATION 

Two  additional  areas  for  possible  application  of  life-cycle  costing  are  (1)  source 
selection  and  (2)  contractual  commitments  in  material  procurement  and  system 
acquisition.  They  are  covered  by  the  following  DoD  documents  (fig  IX-4). 

•  Life  Cycle  Costing  Procurement  Guide  (LCC-1) 

•  Casebook,  Life  Cycle  Costing  in  Equipment  Procurement  (LCC-2) 

•  Life  Cycle  Costing  Guide  for  System  acquisitions  (LCC-3) 


*  USE-CYCLE  COSTING  PROCUREMENT  GUIDE  (INTERIM),  JULY  1970 
••UFE-CYCLE  COSTING  GUIDE  FOR  SYSTEM  ACQUISITIONS  (INTERIM),  DOD.  JANUARY  1973 


Figure  tX-4.  Ufe-cycle  costing  reference  documentation 


LCC-1  addresses  life-cycle  costing  in  the  procurement  of  material  and  hard¬ 
ware  other  than  complete  weapon  systems.  It  presents  a  set  of  general  guidelines 
which  may  be  modified  to  suit  the  needs  of  a  specific  application.  Examples  of  appli¬ 
cations  of  the  general  guidelines  are  given  in  LCC-2.  LCC-3  addresses  life-cycle  cost¬ 
ing  in  the  acquisition  of  material  at  the  complete  system  level.  It  covers  the 
application  of  the  life-cycle  cost  concept  to  the  various  acquisition  strategies  and  con¬ 
tains  an  operating  and  support  cost  model  for  predicting  system  ownership  cost. 


IMPLEMENTING  LIFE-CYCLE  COSTING 


Certain  basic  requirements  or  preparatory  steps  which  will  help  to  assure  use¬ 
ful  results  from  the  application  of  life-cycle  costing  are  described  below. 

PROBLEM  DEFINITION 


Defining  the  problem  is  the  first  step  in  a  life-cycle  cost  analysis.  During  the 
problem  definition  phase,  close  cooperation  between  the  cost  analyst  and  the  system 
engineer  is  required  so  that  the  system  to  be  studied  can  be  adequately  described  for 
costing  purposes.  Before  the  analysis  effort  can  begin,  the  cost  analyst  must  have  (1) 
a  description  of  the  system  to  be  costed  and  (2)  cost  ground  rules  for  the  particular 
study.  As  part  of  the  system  description,  all  relevant  cost-significant  elements  includ¬ 
ing  system  performance  characteristics,  physical  characteristics,  and  operational 
assumptions  —  must  be  identified  and  described.  Since  tho  system  descriptions 
needed  by  the  cost  analyst  can  differ  considerably  from  those  used  by  the  system  engi 
ncer,  direct  communication  between  the  cost  analyst  and  the  system  engineer  is  usu¬ 
ally  necessary.  A  complete  understanding  of  the  ground  rules  is  also  necessary  Exam¬ 
ples  include  the  typo  of  cost  index  to  be  used  (e.g.,  15-year  system  cost),  rules  regard¬ 
ing  amortization  or  discounting,  date  for  which  all  prior  costs  are  to  be  considered  as 
sunk  costs,  and  special  rules  regarding  base  pay  of  operating  personnel  and  attrition 
rates. 

DATA  ACQUISITION 


The  types  of  data  needed  are  identified  from  tho  system  cost  elements  list 
which  is  based  on  the  system  description.  The  data  acquisition  effort  must  begin  long 
before  the  data  are  needed,  and  the  data  must  cover  tho  needs  for  specific  systems, 
including  the  types  of  resources,  cost  equations,  and  cost  factors.  The  data  collected 
should  be  organized,  with  means  for  indexing  and  classifying  cost  and  related  data 
and  means  for  ready  access  to  the  data  by  users. 

DERIVATION  OF  ESTIMATE 


This  is  the  step  in  system  costing  which  is  concerned  with  the  actual  calcula¬ 
tion  of  the  cost  estimates.  The  cost  estimates  are  developed  within  the  framework  of 
cost  element  lists,  the  purpose  of  which  is  to  identify  and  account  for  all  elements  of 
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cost  associated  with  the  system.  The  cost  elements  are  subdivisions  of  the  major  cost 
categories,  including  development,  investment,  and  operation  and  support  costs. 
There  is  no  single  cost  element  list  which  can  serve  the  needs  of  all  problems.  Lists 
have  to  be  flexible  in  their  makeup,  as  they  must  be  adapted  to  the  type  of  system, 
the  nature  of  the  problem,  and  the  type  of  analysis  considered.  The  list  should  high¬ 
light  the  key  features  of  the  system  and  permit  maximum  use  of  the  data  collected. 
Once  a  satisfactory  cost  element  list  is  developed,  calculation  of  cost  estimates  is  ac¬ 
complished  through  cost  estimating  relationships,  cost  models,  or  other  means. 

PRESENTATION  OF  ESTIMATES 


This  step  is  concerned  with  the  communication  of  results  to  the  user,  and  it  is 
related  to  the  first  step,  problem  definition.  For  ease  of  understanding,  the  cost 
estimates  must  be  presented  in  terms  and  formats  as  well  as  at  a  level  of  detail  appro¬ 
priate  for  the  type  of  decision  to  be  made.  The  terms  and  formats  vary  with  the  appli¬ 
cation,  and  they  should  be  given  early  consideration. 

ANALYSIS  DOCUMENTATION 


Proper  documentation  of  the  cost  analysis  is  important  to  both  the  analyst 
and  tho  users  of  the  analysis.  In  addition  to  its  use  for  review  and  evaluation,  the 
documentation  may  sorve  as  a  record  of  the  study  and  a  sourco  of  data  which  can  be 
usod  for  other  studies.  For  review  and  evaluation  purposes,  the  documentation  should 
clearly  describe  and  discuss  tho  procedures  and  data,  as  well  as  the  data  sources  used. 
It  should  also  permit  tho  cost  estimates  to  be  reproduced,  if  necessary,  via  the  proce¬ 
dure  or  process  described. 

REVIEW  AND  EVALUATION 


The  user  has  the  critical  task  of  judging  the  cost  estimates  and  evaluating 
them  as  to  suitability  and  credibility.  Since  direct  verification  for  accuracy  is  imprac¬ 
tical,  particularly  when  cost  estimates  are  used  for  decisions  involving  systems  far 
into  the  future,  emphasis  must  bo  placed  on  evaluating  the  validity  of  the  cost  study 
itself  and  the  analysis  underlying  it.  In  particular,  areas  such  as  data,  data  sources, 
methods,  procedures,  and  conclusions  should  be  closely  reviewed  and  evaluated.  It  is 
by  understanding  the  manner  in  which  the  cost  estimates  are  derived  that  a  measure 
of  confidence  may  be  obtained  for  using  them. 
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X.  INTEGRATED  LOGISTICS  SUPPORT  (ILS) 


The  ILS  functions  are  essential  to  the  successful  integration  of  an  equipment 
into  operational  use.  The  ILS  concepts  must  be  initiated  in  conjunction  with  the 
equipment  design  concept.  All  too  often,  the  consideration  of  the  ILS  is  postponed 
into  the  development  phase  or  later;  at  this  late  stage,  the  ILS  tasks  become  little 
more  than  analysis  of  an  existing  design  to  determine  what  must  be  done  to  support 
it.  The  support  system  resources  are  limited,  and  design  chases  are  expensive  to 
implement.  So,  the  most  effective  way  of  implementing  ILS  is  to  design-in  the  neces¬ 
sary  equipment  features  initially.  This  requires  integrated  planning  of  design  for  per¬ 
formance  and  design  for  support. 

The  ILS  disciplines  include  the  following; 

Reliability 

Maintainability 

Human  engineering 

Safety  engineering 

Transportability 

Personnel  and  training  design 

Test  equipment,  tent  criteria,  facilities,  and  technical  data  planning 

Logistics  design  (including  logistics  support  analysis,  levcl-of-rcpair  analysis, 
and  standardization  engineering) 

Assistance  in  these  disciplines  is  usually  available.  NOSC,  for  example,  has 
specialists  in  the  Product  Assurance  Division  and  the  Design  Engineering  Division. 
Their  participation  should  be  included  to  an  appropriate  degree  in  any  project  devel¬ 
oping  equipments  for  fleet  use.  The  project  manager  should  become  familiar  with  ILS 
concepts;  the  Integrated  Logistics  Support  Implementation  Guide  for  DoD  systems 
and  Equipments  (NAVMAT  P4000)  is  a  roadily  available  reference. 


TIIE  ILS  CONCEPT 


CONCEPT 


The  ILS  concept  is  concerned  with  definition,  optimization,  and  integration  by 
systematic  planning,  implementation,  and  management  of  logistic  support  resources 
throughout  the  system  life  cycle.  The  concept  is  realized  through  the  proper  integra¬ 
tion  of  logistic  support  elements  with  each  other  and  through  the  application  of  logis¬ 
tics  considerations  to  the  decisions  made  on  the  design  of  the  hardware  system  and 
equipment  as  part  of  the  system  engineering  process. 
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OBJECTIVE 


It  has  long  been  obvious  to  those  responsible  for  the  operation  of  military  sys¬ 
tems  and  equipments  that  support  problems  are  a  limiting  factor  on  operational 
capability  availability.  Much  effort  is  expended  in  trying  to  increase  mean  time 
between  failures  or  decrease  periodic  maintenance,  and  to  reduce  maintenance  down¬ 
time  Operational  commanders  watch  carefully  the  statistics  on  those  items  of  equip¬ 
ment  which  are  not  operationally  ready  because  of  maintenance  or  supply  difficulties. 
They  recognize  the  importance  of  having  personnel  who  are  adequately  trained  to 
operate  the  equipment  properly  and  to  maintain  it  efficiently  in  order  to  reduce  the 
number  and  frequent^  of  failures  and  to  reduce  the  adverse  effect  of  such  failures  and 
maintenance  time  on  operational  readiness.  They  know  the  importance  to  their 
operation  of  adequate  facilities  and  support  equipment. 

The  ILS  concept  must  be  applied  throughout  the  acquisition  cycle  to  ensure 
that  systems  are  designed  to  meet  operation  requirements.  Frequently  systematic 
consideration  of  the  solution  to  the  problems  of  support  does  not  begin  until  the  sys¬ 
tem  is  in  the  production/deployment  phase  While  some  elements  of  support  may 
receive  early  attention,  it  is  rare  that  total  support  planning  has  a  mqjor  impact  on 
system  design  This  lack  of  timely  and  systematic  planning  adversely  reflects  opera¬ 
tional  availability  and  cost  of  ownership. 

Under  the  ILS  concept,  the  importance  of  trading  off  operation  and  support 
requirements  from  the  earliest  phases  of  the  life  of  a  system  has  been  recognized.  As 
DoD  Directive  4100.35  states.  “Over  the  life  cycle  of  a  system,  support  represents  a 
major  portion  of  the  total  cost,  and  is  sometimes  the  principal  cost  item  "  By  integra¬ 
tion  of  logistics  considerations  into  the  conceptual  planning  and  through  the  entire 
design  and  development  process,  either  support  costs  during  the  operation  may  be 
significantly  reduced,  or  operational  availability  of  the  system  may  be  increased  with¬ 
out  a  significant  increase  in  cost. 

In  addition  to  integrating  support  planning  into  the  entire  design  and  develop¬ 
ment  process,  it  is  also  fundamental  to  the  ILS  concept  that  the  elements  of  logistic 
support  (as  listed  and  defined  in  appendix  B)  shall  be  integrated  with  each  other  into 
a  total  support  system.  When  the  baseline  of  any  one  logistics  element  is  changed  or 
proposed  to  be  changed,  the  effect  on  all  other  logistics  elements  and  on  the  total  sys¬ 
tem  must  be  formally  considered  and  necessary  adjustments  made. 

In  applying  the  concept  of  ILS  to  a  system/equipment  acquisition,  it  is  impor¬ 
tant  to  maintain  a  proper  perspective  —  to  bear  in  mind  that  logistics  support  is  not 
an  end  in  itself,  but  exists  only  to  support  the  operation  of  tne  system/equipment  to 
which  it  is  related  The  support  problem  will  vary  according  to  the  complexity  and 
value  of  the  system/equipment.  Planning  for  support  must  be  tailored  to  each  acquisi¬ 
tion  individually;  this  guide  addresses  the  differences  of  approach  for  mqjor  acquisi¬ 
tion,  less-than-major  acquisitions,  off-the-shelf  items,  and  modification  programs. 

It  is  also  necessary  to  bear  in  mind  that  in  any  acquisition  which  includes 
development,  there  are  two  entirely  different  types  of  effort,  first  is  the  conceptual 
and  broad  planning  stage;  second  is  the  period  from  full-scale  development  through 
final  disposition,  in  which  the  actions  contemplated  in  the  first  stage  are  refined  and 
implemented.  Just  as  support  planning  must  be  tailored  to  the  type  of  acquisition,  it 
must  also  be  tailored  to  the  time  phasing  of  the  acquisition  process. 
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The  first  part  of  the  logistics  problem  in  an  acquisition  is  to  establish  basic 
characteristics  which  will  enable  the  operational  requirements  to  be  achieved. 

Program  managers  must  keep  the  operational  mission  clearly  in  view  during  the  early 
stages,  and  they  should  recycle  and  refine  their  planning  to  determine  what  is  the 
minimum  which  must  be  accomplished  prior  to  full-scale  development.  Once  the  basic 
logistics  system  characteristics  are  formulated,  they  must  be  stated  to  the  design 
engineers  in  a  “design  to”  or  “design  constraint"  fashion.  When  requirements  are 
stated  in  this  format,  they  may  be  used  in  analytical  and  tradeoff  studies.  In  the 
development  of  the  logistic  support  concepts  and  the  early  planning  for  support,  pro¬ 
gram  managers  must  assure  that  logistic  and  design  personnel  work  together  in  an 
atmosphere  of  maximum  cooperation  and  communication.  Thus,  the  ILS  function 
must  be  closely  identified  as  an  integral  part  of  the  total  system  engineering  process. 

The  logistics  effort  in  the  early  stages  must  be  confined  to  development  and 
formulation  of  inclusive  but  broad  logistics  plans  and  support  characteristics.  The 
result  should  be  a  road  map  of  what  specific  steps  will  be  taken,  at  what  time,  and  in 
what  detail  as  the  development  progresses  and  the  design  matures.  The  detailed  plan¬ 
ning  and  preparation  of  detailed  data  packages  must  be  deferred  until  the  configura¬ 
tion  of  the  hardware  has  been  reasonably  stabilized  Detailed  support  planning  which 
is  accomplished  prior  to  the  establishment  of  the  basic  configuration  and  dependent 
on  that  configuration  is  almost  certain  to  require  extensive  rework  to  become  valid 
and  usable 

The  techniques  for  the  application,  testing,  and  demonstration  oflLS  planning 
and  the  requirements  for  the  management  of  the  logistics  effort  at  various  stages  of 
the  acquisition  process  are  covered  in  greater  detail  in  the  ILS  Implementation  Guide 
(NAVMAT  P4000)  previously  referenced 

RELIABILITY  PROGRAM 

Reliability  is  the  probability  that  an  item  will  perform  its  intended  function 
for  a  specific  interval  within  a  stated  operational  environment  To  achieve  the 
required  level  most  effectively,  the  project  manager  must  establish  a  reliability  pro¬ 
gram  m  the  early  phases  of  the  development  He  must  ensure  that  the  designer 
achieves  the  maximum  m  reliability  whic'-  is  available  to  him;  that  the  most  reliable 
components  available  arc  used;  that  reliability  is  emphasized  in  all  succeeding  phases 
of  development;  and  that  design  revisions  to  incorporate  modern  technology  are 
incorporated  at  the  appropriate  stage  of  maturity.  He  must,  in  short,  program  reli¬ 
ability  from  concept  formulation  to  production. 

A  reliability  program  consists  of  plans  and  tasks  scheduled  in  a  manner  to  pro¬ 
vide  control  over  all  factors  affecting  the  reliability  of  equipment  during  conceptual 
design  and  feasibility  demonstration,  development,  preproduction,  and  production  to 
ensure  that  the  quantitative  reliability  requirements  of  the  equipment  are  met. 

The  project  manager  bases  his  reliability  program  on  two  basic  command¬ 
ments- 

»  Keep  equipment  simple  —  eliminate  the  superfluous. 

•  Determine  minimum  acceptable  performance  levels  —  specify  nothing  that 
exceeds  them. 
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Observance  of  these  commandments  simplifies  design,  fabrication,  operation, 
and  support;  lowers  development,  purchase,  and  support  costs;  and  maximizes  reli¬ 
ability. 


Detailed  reliability  program  requirements  are  provided  in  MIL-STD-785.  See 
figure  X-l. 

THE  RELIABILITY  PROGRAM  PLAN 


Management  Tasks 

The  reliability  program  plan  should: 

•  Identify  the  organizational  elements  and  the  key  personnel  responsible  for 
managing  the  overall  reliability  program. 

•  Clearly  define  the  related  responsibilities  and  functions,  including  both 
policy  and  action. 

•  Stipulate  the  authority  delegated  to  the  organizations  to  enforce  its  poli¬ 
cies. 

•  Identify  the  relationships  between  line,  service,  staff,  mid  policy  organiza¬ 
tions. 

Specific  management  tasks  called  out  in  MIL-STD-785  for  the  reliability  plan 
include: 

A  detailed  listing  and  description  of  each  task. 

A  reference  list  of  detailed  procedures,  with  summary  descriptions,  to  evaluate 
the  status  and  control  of  each  task  including  identification  of  the  organiza¬ 
tional  unit  with  the  authority  and  responsib’iity  for  executing  each  task. 

The  anticipated  man-hours  required  for  each  task. 

Identification  of  known  reliability-oriented  problems  to  be  solved,  an  assess¬ 
ment  of  the  impact  of  these  problems  on  specified  program  requirements,  and 
the  proposed  solutions  or  the  proposed  program  to  solve  these  problems 

Procedures  for  recording  the  status  of  actions  to  resolve  the  problems 

Designation  of  progressive  reliability  value  and  milestones,  definition  of  inter¬ 
relationships,  and  estimation  of  times  needed  for  each  reliability  program  ac¬ 
tivity  or  task.  (Where  PERT  is  used  in  the  total  program,  appropriate  reliabil¬ 
ity  milestones  should  be  included  in  the  overall  network.) 
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Figure  X-1.  Reliability  documents 


Method  by  which  the  reliability  requirements  are  disseminated  to  designers 
and  associated  personnel 

Provisions  for  reliability  indoctrination  and  training  in  connection  with  the 
project. 

Reliability  Tasks 

Reliability  tasks  germane  to  a  reliability  program  are 

Program  planning 

Design  guides 

Mathematical  modeling 

Allocation 

Prediction 

Failure  modes  and  effects  analysis 
Parts  program 
Tradeoff  studies 
Contractor  control 

Documentation  and  data  control  plans 
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Design  reviews 

Developmental  test  planning  and  testing 

Failure  analysis  during  testing 

Most  of  these  tasks  are  performed  during  the  concept  design  and  developmen¬ 
tal  stages  of  a  program  before  a  preproduction  design  is  formulated.  Application  of 
the  tasks  during  each  acquisition  phase  is  described  in  MIL-STD-785.  A  number  of 
these  tasks  are  further  described  below. 

Allocation 

Allocation  is  apportionment  of  equipment  reliability  from  an  individual  equip¬ 
ment  specification  to  lower  limits  within  the  equipment.  This  is  accomplished  by  allo¬ 
cating  numerical  reliability  goals  to  each  assembly  and  subassembly  down  to  each 
nonrepairable  part.  When  combined  with  the  mathematical  model,  the  allocated  goals 
should  yield  an  equipment  reliability  not  less  than  that  required  in  the  individual 
equipment  specification.  A  reliability  group  isually  performs  this  task  and  the  results 
provide  the  project  manager  with  his  numerical  reliability  requirement. 

Prediction 

Prediction  is  the  determination  of  equipment  reliability  from  the  reliability 
characteristics  of  its  components  Reliability  predictions,  performed  early  in  the 
design  phase,  are  used  as  a  basis  for  determining  the  adequacy  of  the  equipment  con¬ 
figurations  and  models  for  meeting  the  allocated  design  goals.  Prediction  makes  it 
possible  to  determine  the  weak  links  in  an  equipment’s  reliability  and  to  determine 
the  necessary  changes,  their  costs,  and  the  reliability  improvement  to  be  expected 
from  their  accomplishment  Subtasks  of  prediction  are.  (1)  reliability  logic  block  dia¬ 
gram  (which  shows  the  relationships  between  equipment  operation/failure  and  con¬ 
stituent  equipment  or  component  operation/failure)  and  (2)  mathematical  model  of 
the  logic  block  diagram  in  equation  form  M1L-STD-75G  shows  how  to  construct  the 
logic  block  diagram  and  make  predictions,  while  M1L-HDBK-217  provides  amplifica¬ 
tion,  elaboration,  and  a  data  base  See  figure  X-l 

Failure  Modes  and  Effects  Analysis  (FMEA) 

FMEA  determines  the  effects  on  hardware  (circuit)  outputs  when  constituent 
parts  fail  in  different  modes.  Typical  part  failure  modes  are  fail  open,  fail  closed,  and 
parameter  drift  Examples  are  given  in  NAVSI1IPS  09G7-31G-8010.  See  figure  X-l. 
This  task  points  out  to  the  project  manager  the  effects  each  item  failure  has  on  the 
design  and  the  manner  in  which  the  item  fails  Failures  which  might  appear  to  be 
simple  could  be  critical  to  system  operation.  This  task  allows  the  designei  to  soften  or 
remove  the  impact  of  the  failure  on  his  design  if  the  item  is  determined  to  be  critical 
to  system  operation.  The  FMEA  results  combined  with  the  prediction  task  will  pro¬ 
vide  information  to  the  project  manager  on  the  reliability  worthiness  of  the  design 
and  the  weak  links  in  the  reliability  chain. 

From  this  information  he  can  make  more  reasonable  decisions  as  to  the  need 
for  design  changes  and,  if  needed,  the  types  or  are  as  of  change  that  will  yield  a  sig¬ 
nificant  improvement  in  reliability.  FMEA  is  used  as  a  tool  for  design  improvements 
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to  eliminate  or  reduce  item  criticality  by  providing  redundancy,  alternate  modes  of 
operation,  or  increased  personnel  and  material  safety. 

Use  MIL-STD-1629  to  perform  FMEA. 

Test  Planning  and  Testing 

The  proof  of  achieved  reliability  and,  conversely,  the  uncovering  of  deficient 
areas  of  design,  lie  in  the  testing  of  the  item.  The  designer  should  gather  appropriate 
data  for  reliability  purposes  during  the  development  and  testing  stages.  Measures 
such  as  accept/reject  criteria,  the  definition  of  a  failure,  and  instrumentation  and 
data  requirements  should  be  established. 

The  designer  should  also  supply  information  which  indicates  the  types  of  tests 
to  specify,  test  equipment  required  for  the  test,  acceptable  limits  of  operation,  and 
type  of  test  report  required  Testing  should  be  performed  at  the  environmental 
stresses  listed  in  the  individual  equipment  specification.  These  inputs  allow  the  test 
and  evaluation  group  to  develop  a  reliability  test  plan  as  described  in  M1L-STD-781. 
See  figure  X-l. 

Failure  Analysis  During  Testing 

This  task  determines  the  following;  (1)  estimates  of  the  reliability  of  hardware 
from  test  data,  (2)  whether  the  equipment  is  to  be  accepted  or  rejected,  and  (3)  causes 
of  failure  and  weaknesses  in  design  (See  M1L-STD-781 )  It  provides  the  basic  data  for 
the  design  analysis  and  redesign  of  equipment  It  provides  the  feedback  loop  to  pro¬ 
ject  managers  so  they  can  effect  a  design  change  that  eliminates  the  uncovered  defi¬ 
ciencies. 

Design  Reviews 

Formal  and  informal  design  reviews  provide  the  necessary  interaction  between 
designers,  project  managers,  sponsors,  and  users  that  permits  an  insight  into  the 
designer’s  ideas  and  allows  an  appraisal  of  his  approach,  progress,  and  problems 
Design  reviews  provide  the  designer  a  more  precise  understanding  of  the  user’s 
requirements  and  problems  and  of  whether  or  not  his  design  approach  will  fulfill  the 
reliability  needs  of  the  user.  Formal  design  review  usually  consist  of  a  Preliminary 
Design  Review,  held  during  the  preliminary  design  of  the  equipment,  the  Critical 
Design  Review,  usually  held  30  days  prior  to  formal  design  release,  and  the  Final 
Design  Review. 

RELIABILITY  REFERENCES 

Naval  Sea  Systems  Command  NAVSH1PS  09G7-LP-597-101 1,  Electronic 

Equipment  Parts  Application,  and  Reliability  Information  for  Navy 

Naval  Ship  Engineering  Center  Military  Standard  MIL-STD-105D,  Sampling 

Procedures  and  Tables  for  Inspection  by  Attributes,  29  Apr  03 

Naval  Air  Systems  Command  Military  Standard  MIL-STD-75GB,  Reliability 

Modeling  and  Prediction,  18  Nov  81 
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Naval  Air  Systems  Command  Military  Standard  MIL-STD-781D,  Reliability 
Testing  for  Engineering  Development,  Qualification,  and  Production,  17  Oct 

Naval  Air  Systems  Command  Military  Standard  MIL-STD-785B,  Reliability 
Program  for  Systems  and  Equipment  Development  and  Production,  15  Sep  80 

Naval  Electronic  Systems  Command  Military  Handbook  MIL-HDBK-217E 
Reliability  Stress  and  Failure  Rate  Data  for  Electronic  Equipment,  20  Sep  74 

Failure  Modes  &  Effects  Analysis,  MIM5TD-1629A,  24  Nov  80 

TR-7,  DoD  Quality  and  Reliability  Assurance  Technical  Report 

MAINTAINABILITY  PROGRAM 


general 

Maintainability  is  that  part  of  equipment  design  which  contributes  to  the 
rapidity  ease  and  economy  of  maintenance  and  repair.  It  provides  design  features 
and  functions  for  simplifying  or  expediting  the  maintenance  tasks  which  must  be 
performed  in  order  to  keep  an  equipment  in  its  specified  operating  condition  or  to 
res  ore  it  to  that  condition  in  the  operating  environment.  A  high  level  of  maintain- 
ability  not  only  reduces  equipment  downtime  but  also  helps  reduce  the  life-cycle  cost 
of  maintenance  and  logistic  support  y 

Early  in  equipment  design,  the  project  manager  arranges  for  a  maintenance 
engineering  analysis  to  be  performed  m  accordance  with  MIL-M-243G5  See  figure 


MIL-STD-1388-1A 

ENGINEERING 

ANALYSIS 


Figure  X-2  Maintainability  documents 


The  project  manager  makes  certain  that  the  designer  is  aware  of  both  the 
established  shipboard  maintenance  procedures  and  the  physical  conditions  under 
which  maintenance  is  to  be  performed;  and  that  he  is  aware  of  the  qualifications  and 
limitations  of  the  technicians  who  will  maintain  the  equipment. 

Early  in  equipment  design,  the  project  manager  arranges  for  a  maintainability 
program  to  be  set  up  in  accordance  with  MIL-STD-470. 

The  maintainability  program  provides  that  the  contractor  must: 

Prepare  a  maintainability  program  plan. 

Perform  a  maintainability  analysis  (distinct  from  the  maintenance  engineering 
analysis).  A  mqjor  task  of  the  maintainability  analysis  is  the  allocation  of 
quantitative  maintainability  requirements  to  all  significant  levels  of  the  sys¬ 
tem. 

Prepare  inputs  to  the  detailed  maintenance  concept  and  detailed  maintenance 
plan. 

Establish  maintainability  design  criteria. 

Perform  design  tradeoffs. 

Predict  maintainability  parameter  values.  Techniques  for  prediction  are  found 
in  MIL-HDBK-472. 

Incorporate  and  enforce  maintainability  requirements  in  subcontractor  and 
vendor  specifications 

Integrate  items  other  than  the  contractor’s  items  into  tho  maintainability  pro¬ 
gram. 

Participate  in  design  reviews  held  at  appropriate  stages  of  the  equipment 
development. 

Establish  a  data  collection,  analysis,  and  corrective  action  system. 

Demonstrate  the  achievement  of  maintainability  requirements.  MIL-STD-471 
gives  the  procedures,  test  methods,  and  requirements  for  verification,  demon¬ 
stration,  and  evaluation  of  the  achievement  of  the  specified  maintainability 
requirement. 

Prepare  maintainability  status  reports  at  determined  or  approved  by  the  pro¬ 
curing  activity. 

MAINTAINABILITY  REFERENCES 

Naval  Air  Systems  Command  Military  Standard  MIL-STD-470A,  Maintain¬ 
ability  Program  Requirements  (For  Systems  and  requirements),  3  Jun  83 

Naval  Air  Systems  Command  Military  Standard  MIL-STD-471A,  Maintain¬ 
ability  Demonstration,  27  Mar  73 
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Naval  Air  Systems  Command  Military  Handbook,  MIL-HDBK-472,  Maintain¬ 
ability  Prediction,  24  May  66 

Naval  Air  Systems  Command  Military  Standard,  MIL-STD-2084,  General  Re¬ 
quirements  for  Maintainability  of  Avionic  and  Electronic  Systems  and  Equip¬ 
ment,  6  Apr  82 

Naval  Ship  Engineering  Center  Military  Standard  MIL-STD-881A,  Work 
Breakdown  Structures  for  Defense  Materiel  Items,  25  Apr  75 

Naval  Sea  Systems  Command  Military  Specification,  MIL-P-24534,  Planned 
Maintenance  Subsystem;  Development  of  Maintenance  Requirement  Cards, 
Maintenance  Index  Pages,  and  Associated  Documentation,  26  Apr  Ti 

HUMAN  ENGINEERING  PROGRAM 


GENERAL 

Electronic  equipment  should  be  designed  to  permit  full  utilization  of  human 
capabilities,  to  compensate  for  human  limitations,  and  to  eliminate  the  possibility  of 
human  error. 

The  project  manager  arranges  for  comments  to  be  solicited  from  operating  and 
maintenance  personnel  during  the  early  planning  stages. 

The  project  manager  arranges  for  operator  interaction  with  equipment  during 
advanced  and  engineering  development  to  aid  in  finding  and  eliminating  hu¬ 
man  engineering  problems  before  the  service  test  demonstration  and  final  pro¬ 
duction  design  specifications. 

The  project  manager  administers  the  human  engineering  program.  He  is  con¬ 
versant  with  the  provisions  of  MILrH-46855,  which  presents  human  engineer¬ 
ing  requirements  including  analysis,  test  and  evaluation,  program  plan,  physi¬ 
cal  models,  procedure  development,  and  coordination  with  other  programs 

The  project  manager  is  also  conversant  with  the  provisions  of  MIL-STD-1472, 
which  presents  human  engineering  design  criteria.  See  figure  X-3. 


|->-  M1L-HDBK-759 
(->-  MIL-HDBK-761 
MIL-HDBK-7S3 

Figure  X-3  Human  engineering  documents. 
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HUMAN  ENGINEERING  REFERENCES 


Naval  Air  Systems  Command  Military  Specification  MIL-H-46855A,  Human 
Engineering  Requirements  for  Military  Systems,  Equipments  and  Facilities, 

2  May  72 

Naval  Air  Systems  Command  Military  Standard  MIL-STD-1472D,  Human 
Engineering  Design  Criteria  for  Military  Systems,  Equipment  and  Facilities, 

14  Mar  89 

US  Army  Military  Handbook  MIL-HDBK-759A,  Human  Factors  Engineering 
Design  for  Army  Material,  30  Jun  81 

Department  of  Defense  DoD-HDBK-7Gl,  Human  Engineering  Guidelines  for 
Management  Information  Systems,  28  Jun  85 

Department  of  Defense  DoD-HDBK-7G3,  Human  Engineering  Procedures 
Guide,  27  Feb  87. 


SAFETY  PROGRAM 


GENERAL 


The  chief  objectives  of  the  safety  program  are  the  prevention  of  injury  to  per¬ 
sonnel  and  tho  prevention  of  damage  to  equipment. 

The  project  manager  determines  whether  not  to  specify  that  the  equipment 
contractor  develop  and  maintain  an  effective  system  safety  program  as  out¬ 
lined  in  MIL-E-16400.  Preparation  of  the  program  is  covered  in  MIL-STD-882 
The  primary  reference  for  safety  criteria  is  requirement  1  of  M1L-STD-454 
See  figure  X-4. 

Tho  project  manager  is  willing  to  expose  himself  to  the  worst-case  personnel 
hazards  the  equipment  he  is  responsible  for  is  capable  of  presenting  to  Navy 
personnel. 


Mll-E-16400  - 

PROGRAM  ^ 


Mll-STD-454 
REO,  1 
CRITERIA 


Mli.-STD.882 

PROGRAM  REQUIREMENTS 


Figure  X-4.  Safety  documents 
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Whether  or  not  a  formal  system  safety  program  is  established  for  the  project,  safety 
should  be  an  issue  integral  to  all  design  reviews  The  equipment  design  and  opera¬ 
tional  procedures  should  be  scrutinized  for  conformance  to  safety  criteria  The  ac¬ 
tions  considered  should  include,  but  not  be  limited  to,  the  following; 

1.  Avoiding,  eliminating,  or  reducing  identified  hazards  by  analysis,  design 
selection,  material  selection,  or  substitution 

2  Controlling  and  minimizing  hazards  to  personnel,  equipment,  and  material 
which  cannot  be  avoided  or  eliminated 

3.  Isolating  hazardous  substances,  parts,  and  operations  from  other  activi¬ 
ties,  areas,  personnel,  and  incompatible  materials 

4.  Incorporating  “failsafe”  principles  where  failures  would  disable  the  system 
or  cause  a  catastrophe  through  injury  to  personnel  or  damage  to  equip¬ 
ment 

5  Locating  equipment  parts  so  that  access  to  them  by  personnel  during  opera¬ 

tion,  maintenance,  repair,  or  adjustment  shall  not  require  exposure  to  haz¬ 
ards  such  as  chemical  burns,  electrical  shock,  cutting  edges,  sharp  points, 
or  toxic  atmospheres 

6  Avoiding  undue  exposure  of  personnel  to  physiological  and  psychological 
stresses  which  might  cause  errors  leading  to  mishaps 

7  Providing  warning  and  caution  notes  in  operations,  assembly,  mainte¬ 
nance,  and  repair  instructions;  and  distinctive  markings  on  hazardous 
parts  equipment,  or  facilities  for  personnel  protection 

8  Minimizing  damage  or  injury  to  personnel  and  equipment  in  the  event  of 
an  accident 

In  satisfying  safety  requirements,  the  design  solutions  should  follow  this  order  of 
precedence. 

1.  Design  for  minimum  hazard 

2.  Safety  devices 

3.  Warning  devices 

4  Special  procedures 

MIL-E-1G400  specifies  the  following  tasks  (they  are  worthy  of  consideration  even 
when  MIL-E-16400  does  not  apply): 

1.  Safety  testing 

2.  Design  review  using  a  system  safety  checklist 

3  Safety  analyses 

a.  Conceptual  safety  analysis  (CSA) 
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b.  Design  safety  analysis  (DSA) 

c.  Functional  safety  analyses  considering,  as  a  minimum. 
Installation  requirements 

Testing  requirements 

Operating  and  maintenance  requirements 

Safety  supervision 

Handling  requirements 

Training  requirements 

d.  Hazard  Mode  and  Effects  Analysis  (HMEA) 

4.  Integration  with  other  project  tasks 

SAFETY  REFERENCES 


Naval  Ship  Engineering  Center  Military  Specification  MIL-E-16400H  (NAVY), 
Electronic,  Interior  Communication  and  Navigation  Equipment,  Naval  Ship 
and  Shore,  General  Specification  for,  24  Dec  74 

Naval  Air  Systems  Command  Military  Standard  MIL-STD-454K,  Standard 
General  Requirements  for  Electronic  Equipment,  31  Aug  73 

Naval  Air  Systems  Command  Military  Standard  MIL-STD-882B,  System 
Safety  Program  for  Systems  and  Associated  Subsystems  and  Equipment: 
Requirements  for,  15  Jul  69 


PACKAGING,  HANDLING,  STORAGE,  AND 
TRANSPORTABILITY  (PIIST)  PROGRAM 


GENERAL 

The  PHST  program  considers  all  the  problems  of  transporting  the  system  or 
equipment  from  the  development  site  to  the  test  site,  from  the  production  line  to 
storage,  and  from  storage  to  service  use,  MIL-STD-1367  contains  the  PHST  program 
requirements.  The  program  can  be  viewed  as  being  in  two  phases  —  analysis  and 
design  The  analysis  phase  determines  what  transportation,  handling,  and  storage  ac¬ 
tions  will  be  used  through  the  life  of  the  equipment.  The  design  phase  determines  the 
detailed  PHST  requirements  and  develops  testing,  packaging,  and  handling  proce¬ 
dures  for  use  with  the  equipment  as  well  as  incorporating  design  features  into  the 
equipment  to  facilitate  the  procedures.  Figure  X-5  shows  the  relationships  of  the  pri¬ 
mary  reference  documents. 
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MIL-STD-1319 

CRITICAL  CHARACTERISTICS 


REQUIREMENTS 


Figure  X-5.  PHST  documents 


REFERENCES 

DoD  Military  Std,  MIL-STD-1791,  Designing  for  Internal  Aenal  Delivery  in 
Fixed  Wing  Aircraft,  13  Oct  85 

DoD  Military  Std,  MIL-STD-13G7,  Packaging,  Handling,  Storage,  &  Trans¬ 
portability  Program  Requirements,  27  Apr  77 

DoD  Military  Std,  MIL-STD-13GGB,  Packaging,  Handling,  Storage,  &  Trans¬ 
portation  System  Dimensional  Constraints,  15  May  81 

DoD  Military  Std,  MIL-STD-13G5A,  General  Design  Criteria  for  Handling 
Equipment  Associated  with  Weapons  and  Weapon  Systems,  20  Jul  81 

DoD  Military  Std,  M1L-STD-794E,  Procedures  for  Packaging  and  Packing  of 
Parts  and  Equipment,  1G  Jul  82 

DoD  Military  Std,  MIL-STD-810D,  Environmental  Test  Methods,  31  Jul  8G 

US  Air  Force  Military  Std,  MIL-P-9024G,  Packaging,  Handling,  and  Trans¬ 
portability  in  System/ Equipment  Acquisition,  G  Jun  72 

Naval  Air  Engineering  Center  Military  Spec,  MIL-P-11GJ,  Methods  of  Preser¬ 
vation  Packaging,  8  Apr  88 

DoD  Military  Std,  MIL-STD-1319A,  Item  Characteristics  Affecting  Transport¬ 
ability  and  Packaging  and  Handling  Equipment  Design,  21  Aug  74 

Naval  Ship  Engineering  Center  Military  Spec,  M1L-E-17555H,  Packaging  and 
Packing  of  Electronic  and  Electrical  Equipment,  Accessories,  and  Provisioned 
Items  (Repair  Parts) ,  15  Nov  84 
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PERSONNEL  AND  TRAINING  PROGRAM 


GENERAL 


The  personnel  and  training  program  implements  the  “human  equation’  of  the 
operation  and  maintenance  concepts.  The  program  starts  with  an  analysis  of  O&M 
concepts  to  determine  the  skills  and  skill  levels  needed  to  support  the  system.  Reli¬ 
ability  program  inputs  help  to  establish  the  man-hour  demands  for  each  skill  The 
man-hour  demands  are  used  to  determine  whether  additional  personnel  will  be 
required  in  the  user  organization  and  at  support  facilities.  The  user  and  support  or¬ 
ganizations  are  then  compared  to  the  projected  skill  requirements  to  determine  tenta¬ 
tive  training  requirements. 

A  training  program  plan  is  then  prepared  in  accordance  with  OPNAV1NST 
1500.8  series.  OPNAVINST  1500.8  and  OPNAVINST  1500.44  together  establish  the 
procedures  for  coordinating  project  requirements  with  the  training  commands.  It 
takas  1  year  or  more  to  establish  the  final  training  requirements  and  2  years  to 
budget  and  to  start  training  implementation.  The  training  plan  also  addresses 
interim  training  measures  for  teams  to  support  test  and  evaluation  nnd  for  instruc¬ 
tors  and  the  initial  students  to  cover  the  gap  between  the  equipment  IOC  and  the  es- 
tablislmient  of  the  final  training  support. 

OPNAVINST  1500.2  dictates  the  policies  and  procedures  for  the  “factory’ 
training  programs  which  are  normally  needed  to  meet  interim  trnining  requirements 
Detailed  training  plans,  course  plans,  training  aids,  and  course  materials  are  pre¬ 
pared  in  accordance  with  MIL-STD-1379.  Interim  training  is  an  essential  project  task 
which  must  be  fully  integrated  into  the  overall  project  plans,  budgets,  and  schedule 
This  tra  ning  provides  key  support  to  final  operational  testing  and  for  phasing  the 
equipment  into  service  use. 


REFERENCES 


US  Navy  Military  Std,  MIL-STD-001379C,  Military  Training  Programs, 

10  Dec  76 

OPNAVINST  1500.8M,  Preparation  and  Implementation  of  Navy  Training 
Plans  (NTPs)  in  Support  of  Hardware  and  Non-hardivare  Oriented  Develop¬ 
ments 

Bureau  of  Naval  Personnel  NAVPERS  18068D,  Manual  of  Qualifications  for 
Advancement,  Jan  77 

Naval  Education  and  Training  Command  NAVEDTRA  10,500,  Catalog  of 
Navy  Training  Courses,  (updated  frequently) 

Bureau  of  Naval  Personnel  NAVPERS  16103C,  Manual  for  Navy  Instructors, 
1964 
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TEST  &  MEASUREMENT  PROGRAM 


GENERAL 


The  T&M  program  encompasses  a  broad  base  of  tasks  required  to  support  the 
maintainability  program.  These  tasks  include  conducting  tradeoff  studies  between 
test  and  measurement  metho  Is,  selecting  test  equipments,  coordinating  the  equip¬ 
ment  design  to  incorporate  test  features  and  test  points  adequate  for  maintenance, 
and  generating  and  documenting  test  procedures  for  organizational  and  depot  use. 
The  T&M  program  and  the  maintainability  program  are  mqjor  determinants  of  the 
maintenance  design  and  of  the  effectiveness  of  the  maintenance  concept  implementa¬ 
tion. 


The  primary  standards  used  by  the  T&M  program  are  M1L-STD-2165,  MIL- 
STD-1388-1,  and  MIL-STD-415.  Tradeoffs  are  established  between  different  test  meth¬ 
ods  using  maintenance  time,  calibration  requirements,  skill  requirements,  equipment 
reliability,  and  cost  criteria.  The  final  design  may  incorporate  any  or  all  of  the  follow¬ 
ing  general  methods: 

Automatic  Test  Equipment  (ATE) 

Built-In  Test  Equipment  (BITE) 

Test  Points  with  General-Purpose  Electronic  Test  Equipment  (GPETE) 

Test  Points  with  Special-Purpose  Electronic  Test  Equipment  (SPETE) 

The  use  of  general-purpose  ATE  systems  such  as  the  Versatile  Avionics  Shop  Tester 
(VAST)  or  tho  Integral  Sensor  Test  System  (ISTS)  may  be  dictated  by  platform  main¬ 
tenance  requirements.  The  use  of  software  diagnostics  within  the  system  m  _  be  very 
desirable.  There  is  no  best  solution  for  all  cases,  so  these  project  tasks  are 
important.  In  each  case,  special  design  provisions  will  be  dictated  by  the  test  methods 
employed.  The  impacts  on  training,  technical  manual,  logistics,  and  facility  require¬ 
ments  need  to  be  considered  and  coordinated  with  those  ILS  tasks.  Unless  justified  by 
other  characteristics,  the  T&M  program  will  endeavor  to  eliminate  special  tools,  test 
equipment,  and  facilities. 

REFERENCES 

DoD  Military  Std,  MIL-STD-415D,  Design  Criteria  for  Test  Provisions  for 
Electronic  Systems  and  Associated  Equipment,  1  Oct  G9 

Department  of  the  Navy  Military  Std,  MIL-STD-1345B,  Test  Requirements 
Document,  Preparation  of,  10  Feb  81 

Naval  Ship  Engineering  Center  Military  Std,  MIL-STD-1326,  Test  Points,  Test 
Point  Selection,  and  Interface  Requirements  for  Equipments  Monitored  by 
Shipboard  On-Line  Automatic  Test  Equipment,  15  Jan  68 
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US  Navy  Military  Std,  MIL-STD-1364C,  Standard  General  Purpose  Electronic 
Test  Equipment,  30  May  75 

DoD  Military  Std,  MIL-STD-2165,  Testability  Program  for  Electronics  Systems 
and  Equipment,  26  Jan  85 

LOGISTICS  PROGRAM 


GENERAL 


The  logistics  program  consists  of  three  subprograms  —  the  support  analysis 
program,  the  provisioning  program,  and  the  standardization  program.  The  purpose  of 
the  logistics  program  is  twofold: 

•  To  drive  the  equipment  design  toward  the  most  economic  methods  of  sup¬ 
port 

•  To  plan  and  to  implement  the  effective  support  of  each  equipment  compo¬ 
nent 

The  many  tasks  of  the  logistics  program  are  so  intertwined  with  the  other  ILS  pro¬ 
grams  that  it  is  normally  managed  directly  by  the  project  ILS  manager.  Like  the 
other  ILS  specialties,  the  logistics  program  can  be  very  complex  or  relatively  simple, 
and  it  should  be  tailored  to  the  project  requirements  The  extent  of  the  support  analy¬ 
sis  program  is  especr'ly  susceptible  to  tailoring  techniques.  However,  the  goals  of 
logistic  program  must  be  satisfied  in  order  to  transition  the  equipment  into  service 
use. 


The  standardization  tasks  are  so  intimately  connected  with  the  management 
of  the  design  effort  that  they  are  provided  separate  treatment  in  chapter  XIV,  Consid¬ 
eration  of  Standardization.  Also,  parts  selection  and  management  concepts  are  dis¬ 
cussed  in  chapter  XXIII.  Refer  to  these  sections  for  a  detailed  discussion  of  standardi¬ 
zation  program  tasks  and  issues 

SUPPORT  ANALYSIS  PROGRAM 


The  support  analysis  program  verifies  the  general  support  concepts  and  syn¬ 
thesizes  detailed  support  plans  by  analyzing  the  requirements  imposed  by  opera¬ 
tional,  design,  and  the  other  ILS  programs.  Support  effectiveness  and  costs  are  the 
maw  decision  factors,  life-cycle  costs  comprise  90%  of  the  cost  factors  In  the  conduct 
of  the  analysis  program,  alternative  methods  of  support  are  v estimated,  and  the  im¬ 
plications  on  designs  are  fed  back  to  the  system  engineer.  System  engineering  con¬ 
ducts  the  necessary  trade  studies  and  either  imposes  design  changes  or  design  freezes 
sufficient  to  establish  the  support  concept  baseline. 

There  are  two  primary  analysis  techniques  employed  in  the  support  analysis 
program:  the  logistic  support  analysis  (LSA)  and  the  level  of  repair  (LOR)  analysis 
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The  LSA  is  a  readily  tailored  management  technique  for  coordinating  ILS  functions 
as  well  as  accomplishing  the  support  analysis  itself.  MIL-STD-13G9  contain  LSA 
requirements,  but  a  more  detailed  breakdown  of  tasks  and  procedures  is  documents  in 
MIL-STD-1388.  The  LOR  analysis  is  a  life-cycle-cost  based  technique  for  establishing 
the  best  location  for  accomplishing  repair  tasks.  MIL-STD-1390  contains  the  detailed 
formulas  used  for  LOR  analysis.  Many  of  the  LOR  inputs  come  from  LSA  computa¬ 
tions,  and  the  LOR  output  is  used  in  later  LSA  tasks. 

PROVISIONING  PRr  AM 

The  provisioning  program  provides  for  the  detailed  analysis  of  each  repair  part 
support  requirement  and  documents  the  results  in  the  appropriate  lists  and  codes  for 
by  the  supply  system.  The  support  analysis  program  will  determine  the  general  spar¬ 
ing  philosophy,  but  the  provisioning  program  fills  out  the  details  The  various  codes 
assigned  to  each  part  will  determine  the  cost  and  supply  effectiveness  of  supporting 
that  part  The  provisioning  program  attempts  to  maximize  the  support  effectiveness 
for  the  equipment  while  minimizing  the  costs  of  inventories.  These  goals  are  greatly 
facilitated  by  effective  standardization  program  and  by  designs  which  incorporate 
built-in  spares.  The  provisioning  program  also  prepares  plans  for  the  provisioning 
conference  (see  chapter  VIII)  and  plans  for  interim  support  until  the  normal  support 
system  is  established.  The  interim  support  plans  may  include  warranty  provisions 
and  contractor  support  and  will  describe  the  controls  and  procedures  for  monitoring 
the  support  In  the  past,  each  service  has  had  its  own  unique  procedures  for  provi¬ 
sioning.  However,  many  standard  pints  are  now  managed  by  the  Defense  Supply 
Agency  or  are  supported  by  a  lead  service  Therefore,  uniform  procedures  have  been 
imposed  for  all  new  equipments  and  systems  through  MIL-STD-15G1.  MIL-STD- 
1388-2  establishes  the  formats  and  preparation  instructions  for  provisioning  technical 
documentation.  MIL-STD-1517  prescribes  the  procedures  and  conditions  for  the 
phased  provisioning  of  interim  and  initial  spares  and  repair  parts.  The  coding  of  re¬ 
pair  parts  is  specified  by  MIL-STD-789 

The  most  challenging  tasks  of  the  provisioning  program  involve  investigating 
and  recommending  special  support  actions  These  include  spare  support  during  devel¬ 
opment  and  methods  of  supply  for  parts-peculiar,  critical  technology  parts,  and  long- 
le^d-time  items  Spare  support  during  development  is  normally  very  low  risk  and  is 
usually  accomplished  by  purchasing  some  comfortable  number  of  spares  over  known 
requirements  Special  methods  of  supply  may  require  purchasing  all  the  life-cycle  re¬ 
quirements  at  once  and  setting  up  a  central  stock  point  (life-of-type  (LOT)  stocking) 
or  designating  a  support  manager  to  store,  manufacture,  or  assemble  repair  items 
from  standard  or  commercially  available  parts 

REFERENCES 

Naval  Electronic  Systems  Command  Military  Standard  M1L-STD-1369,  Inte¬ 
grated  Logistic  Support  Requirements,  31  Mar  71 

DoD  Military  Standard  MIL-STD-1388,  Logistic  Support  Analysis,  15  Oct  73 

Naval  Weapons  Engineering  Support  Activity  Militaiy  Standar  d  MIL- 

STD-1390a,  Level  of  Repair,  8  Jul  88 
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GREENING  TECHNIQUES 


XI.  SCREENING  TECHNIQUES 


INTRODUCTION 


The  development  process  is  an  expensive  and  time-consuming  way  to  acquire 
equipment;  wherever  an  existing  equipment  can  be  utilized  off-the-shelf  or  after 
incorporating  a  simple  modification,  it  will  likely  prove  to  be  a  more  cost-effective 
3Da‘Ive*!“?“  soy  development  alternative.  However,  there  are  some  important 
tradeoffs  which  must  be  evaluated  to  establish  this  fact  for  each  specific  acquisition 
These  tradeoffs  imply  questions  which  include  the  following;  q 

1.  What  additional  functions,  performance  levels,  or  features  are  valuable  to 
incorporate  over  the  minimum  requirements?  (It  is  likely  that  existing 
equipments  will  lack  those  features  which  might  extend  the  practical  serv- 
simpler)  ^  eqU'pment  siSnificantly  or  make  operation  and  maintenance 

2.  Will  existing  equipments  be  supportable  in  the  field? 

3’  StS? <X1UipmentS  b*  aVailabIe  in  the  future  t0  support  future  acqui- 

T.be  efTft.^e“r,een  obtains  the  information  necessaiy  to  answer  these  ques- 
‘  !  10  ,?tabllsh  th*  tradeoff  decision  points  between  development  and  off-the- 

shelf  alternatives,  even  if  development  is  indicated,  a  great  deal  of  valuable 
information  can  be  gained  through  the  screen  help  avoid  subtle  pitfalls 

A  screen  consists  of  the  following  elements' 

•  Locating  candidate  equipments 

•  Eliminating  unlikely  candidates 

•  Conducting  visual  inspections 

manuaIs'  0Perating  instructions,  and  other  data  asso¬ 
ciated  with  the  equipment 

•  Contacting  manufacturers,  dealers,  and  users 

•  Conducting  performance  and  environmental  tests 

„Tba  ooroen  depends  heavily  on  adequately  stated  technical  requirements  for  its 
success.  Usually  he  screen  will  not  produce  a  black-and-white  decision  so  a" 
nomic  analysis  will  be  necessaiy  to  reach  a  final  conclusion.  bon'som  “0 


CONSTRUCTING  THE  SCREEN 


The  screen  is  derived  directly  from  acquisition  requirements  consisting  of: 

•  Technical  requirements  contained  in  the  system  specification 

•  Procurement  requirements  (the  number  of  units  needed  in  the  near  term 
and  in  the  future) 

•  Peculiar  features  of  the  intended  application  (see  the  five  major  issues  dis¬ 
cussed  in  chapter  VI,  Transition  to  Production) 

•  Support  requirements 

•  Cost  constraints 

The  technical  requirements  must  fully  embrace  the  interfaces,  human  engi¬ 
neering  criteria,  reliability,  maintainability,  safety,  transportability,  environmental 
conditions,  and  so  forth  which  are  dictated  by  the  operational  requirements  Nor¬ 
mally,  the  system  specification  is  not  adequately  formulated  until  near  the  end  of  the 
conceptual  phase  and  not  validated  until  the  validation  phase  is  nearly  complete  The 
screen  can  be  applied  in  the  conceptual  phase,  but  not  without  some  risk  resulting 
from  inadequate  technical  requirements  The  end  of  the  validation  phase  is  the  rec¬ 
ommended  time  to  implement  the  screen,  because  the  technical  requirements  must  be 
grouped  into  “hard”  requirements  and  “soft"  requirements  Hard  requirements  are  all 
the  characteristics,  functions,  and  parameters  which  are  essential  in  the  equipment, 
hard  parameters  must  be  toleranced.  Soft  requirements  consist  of  the  desirable  char¬ 
acteristics  and  functions  and  of  the  parameters  which  may  be  “sloppy  ’  It  may  be  dif¬ 
ficult  to  group  requirements  prior  to  the  validation  phase  The  acquisition 
requirements  can  now  be  applied  to  the  screening  steps  in  the  manner  described  in 
succeeding  sections, 

LOCATING  CANDIDATE  EQUIPMENTS 


Candidate  military  equipments  can  be  obtained  by  determining  the  proper 
nomenclature(s)  for  the  desired  function  using  MIL-HDBK-140  and  then  searching 
these  nomenclatures  in  the  Joint  Electronics  Type  Designation  File  (available  on  mi¬ 
crofilm  in  the  Library).  The  alphabetical  listing  of  military  specifications  may  also 
prove  useful  Some  candidates  may  be  rejected  and  new  ones  found  through  discus¬ 
sions  with  cognizant  personnel  in  the  appropriate  Naval  Systems  Command,  the 
Army  Electronics  Command,  or  the  Air  Force  Electronics  Command  The  GSA  cata¬ 
log  also  lists  possible  candidates. 

Candidate  commercial  equipments  can  be  isolated  through  the  Design  Engi¬ 
neering  File  of  the  Visual  Search  Microfilm  File  (VSMF)  service,  which  is  a  compen¬ 
dium  of  supplier  catalogs.  Other  candidates  may  be  supplied  by  the  appropriate 
technical  or  industrial  society  and  by  known  commercial  users  of  like  equipment.  The 
Conover-Mast  Purchasing  Directory  (a  Cahners  publication),  Defense  Marketing 
Services,  the  Directory  of  Engineering  Document  Sources  (Global  Engineering 
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Document  Services  publication),  Electronic  Industry  Telephone  Directory  (Harris 
Publishing  Company,  Cleveland,  Ohio),  and  the  Thomas  Register  are  also  useful.  For 
items  over  $10,000,  a  “sources  sought'  can  be  issued  in  the  Commerce  Business  Daily 
(CBD);  the  CBD  advertisement  should  specifically  restrict  respondees  to  existing 
products  when  it  is  used  for  this  purpose. 

In  either  case,  the  “hard"  technical  requirements  are  the  governing  criteria  for 
conducting  the  search.  When  sufficient  information  is  not  available,  the  manufacturer 
should  be  queried  directly;  however,  it  is  necessary  to  take  care  not  to  disclose  infor¬ 
mation  which  would  give  the  manufacturer  an  unfair  advantage  should  a  develop¬ 
ment  be  necessary  (to  do  so  is  a  violation  of  the  Armed  Forces  Procurement 
Regulations). 


ELIMINATING  UNLIKELY  CANDIDATES 


The  following  candidates  should  be  eliminated: 

1.  Any  candidate  not.meeting  the  hard  technical  requirements  which  cannot 
easily  be  modified  tc  do  so. 

2.  Any  candidate  which  cannot  be  supported  adequately  in  the  field  (this  nor¬ 
mally  applies  only  in  situations  where  depot-level  maintenance  cannot  be 
employed). 

3.  Any  candidate  which  is  not  truly  off-the-shelf.  Some  military  equipments 
are  available  only  after  a  sufficient  quantity  is  needed  and  must  be  “rede¬ 
veloped”  for  each  buy.  Some  commercial  manufacturers  advertise  and  take 
orders  for  equipment  still  under  design;  the  design  may  be  canceled  due  to 
lack  of  interest  or  altered  by  a  large  user  with  substantially  different 
needs. 

4.  Candidates  not  conforming  to  the  Buy  American  Act  (41  USC  10B(b)) 
unless  there  is  no  domestic  candidate  available,  the  item  is  for  use  in  the 
country  of  origin,  or  the  item  is  a  Canadian  Military  Supply  (such  supplies 
are  exempted  from  the  Buy  American,  Act;  see  ASPR  6-103.5(a)  and  NPD 
6-103.5).  (Many  NATO  sources  are  exempted  from  the  Buy  American  Act.) 

5.  Candidates  obviously  not  meeting  project  cost  constraints. 

If  a  substantial  number  of  candidates  remain,  the  list  can  be  further  narrowed 
by  requiring  further  compliance  to  soft  technical  requirements.  Also,  it  is  desirable  to 
eliminate  unreliable  manufacturers;  however,  this  action  must  be  in  compliance  with 
ASPR  Section  1,  Part  6  (1  July  1974). 

Prepare  a  priority  list  of  candidates  based  upon  their  conformance  to  all  acqui¬ 
sition  requirements,  their  success  as  products  (talk  to  users  and  distributors),  their 
apparent  ability  to  perform  in  applications  closely  related  to  the  intended  use,  the 
availability  of  long-term  warranties  on  them,  and  other  general  indicators  of  rugged¬ 
ness,  reliability,  availability,  and  suitability. 
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CONDUCTING  VISUAL  INSPECTIONS 


Candidate  equipments  should  be  inspected  visually  for  workmanship,  enclo¬ 
sure  effectiveness,  human  engineering,  safety  design,  maintainability,  operability,  and 
thoughtfulness  and  vintage  of  design.  Mqjor  and  minor  discrepancies  should  be  noted. 
Major  discrepancies  are  cause  for  rejection  if  they  cannot  be  corrected  through  simple 
modifications.  An  archaic  design  may  be  cause  for  rejection  unless  the  manufacturer 
can  guarantee  supply  parts  support.  Too  new  a  design  (experimental  or  unproved) 
should  be  rejected  unless  the  manufacturer  is  willing  to  offer  a  long-term  warranty. 


REVIEWING  DATA 


All  available  specifications,  technical  manuals,  instructions,  schematics,  and 
other  equipment  data  should  be  reviewed  to  determine  whether  they  are  adequate  to 
support  the  project  needs.  Technical  manuals  should  meet  the  standards  of  MIL-M- 
7298,  Manuals,  Technical,  Commercial  Equipment.  Obtain  copies  of  schematics, 
parts  lists,  and  technical  manuals  wherever  possible.  Determine  if  there  are  any 
deviations  or  waivers  of  specifications,  especially  for  military  items.  Review  the  sche¬ 
matics  for  apparent  capability  of  meeting  the  specification  and  interface  require¬ 
ments.  Review  the  parts  list  to  determine  the  reliability  and  availability  of  the  parts. 
Review  the  operations  and  maintenance  procedures  to  determine  their  clarity  and 
completeness. 

Using  a  combination  of  hardware  and  documentation,  evaluate  the  following 

items: 

•  Net  input  power/volume  (an  indication  of  heat  buildup). 

•  Operating  temperature  (an  indication  of  component  design  limitations  and 
humidity  resistance). 

•  Cooling  capability/net  input  power  (an  indication  of  cooling  effectiveness). 

•  Internal  voltage  levels  (another  indication  of  humidity  resistance). 

•  Equipment  weight,  volume,  shape,  and  mounting  provisions  (indicators  of 
shock  and  vibration  resistance,  and  interface  or  installation  problems). 

•  Manufacturer’s  claimed  environmental  specifications,  if  any. 

•  Component  weight,  volume,  shape,  and  mounting  provisions  (further  indi¬ 
cators  of  shock  and  vibration  resistance). 

•  Electronic  component  count  (an  indication  of  reliability,  maintainability, 
and  cost  of  logistics  support).  (Mechanical  parts  also  fail,  but  sue.  failures 
tend  to  be  negligible  in  electronics  equipment.  A  relative  parts  count 
between  competing  equipments  cannot  tell  as  much  as  a  weighted  parts 
count  based  on  parts  failure  rates.  However,  all  that  is  necessary  the  first 
time  through  this  step  is  a  rough  approximation.) 
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CONTACTING  MANUFACTURERS,  DEALERS, 
AND  USERS 


Sales  brochures  and  data  sheets  often  simplify  or  exaggerate  specifications. 
They  may  make  claims  that  cannot  be  substantiated.  Performance  specifications  may 
apply  only  to  limited  circumstances  and  ideal  conditions,  or  may  be  valid  only  with 
special  options.  Certain  specified  performance  levels  may  not  be  realizable  simultane¬ 
ously. 

•  Contact  the  manufacturer’s  marketing  an  engineering  departments.  Dis¬ 
cussions  with  technical  experts  from  the  company  may  reveal  hidden 
defects  or  additional  capabilities  not  apparent  from  sales  literature.  New 
products  may  be  in  production.  Perhaps  variations  of  the  advertise  equip¬ 
ment  or  special  options  are  available. 

•  Ask  the  manufacturer  to  explain  specifically  why  his  product  is  better  than 
the  others  under  consideration.  An  alert  manufacturer  knows  his  competi¬ 
tion  and  is  eager  to  show  off  his  product’s  advantages.  Such  comparative 
information  will  point  out  subtle  problems  not  readily  visible  in  product 
sales  literature  and  not  likely  to  be  raised  by  a  manufacture  with  regard  to 
his  own  product. 

•  Sift  through  the  comparisons  furnished  by  all  companies.  Give  each  manu¬ 
facturer  a  chance  to  defend  his  equipment  or  to  offer  solutions  to  any  seri¬ 
ous  problems  pointed  out  by  competitors,  without  identifying  the  source  of 
your  information. 

•  Obtain  the  names  of  dealers  who  supply  the  product.  Solicit  dealer  com¬ 
ments  on  the  suitability  of  his  equipment  for  the  application,  his  experi¬ 
ence  with  the  manufacturer,  and  customer  comments  or  complaints  he  has 
received.  Obtain,  if  possible  (from  dealer  or  manufacturer),  a  history  of  the 
item’s  usage  and  failure,  and  a  list  of  purchasers. 

•  Request  information  from  purchasers  regarding  performance,  problems,  or 
failures  versus  usage.  Determine  whether  the  purchasers’  mounting  loca¬ 
tions,  operational  requirements,  and  environments  are  comparable  to  those 
of  the  intended  application  so  that  these  comments  can  be  effectively 
evaluated. 

•  Interrogate  the  Government  Industry  Data  Exchange  Program  (GIDEP) 
for  information  on  each  product  (if  any  is  available). 

OBTAINING  UNITS  FOR  TEST 


Usually,  it  is  not  necessary  to  actually  acquire  equipments  until  the  perform¬ 
ance  and  environment  tests  are  conducted.  The  equipments  can  often  be  viewed  and 
inspected  at  a  user’s  facility  or  the  manufacturer’s  plant.  However,  the  equipments 
must  be  “in  hand”  for  testing. 
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If  the  procurement  requirements  offer  the  prospective  suppliers  the  promise  of 
reasonably  large  sales,  there  will  be  sufficient  incentive  to  the  supplier  to  make  a  unit 
or  two  available  (on  loan)  for  testing.  If  a  formal  “sources  sought”  has  been  used  to 
find  candidates,  and  if  it  appears  reasonably  certain  one  or  more  candidates  will  be 
acceptable,  a  bid  sample  can  be  requested.  In  either  case,  equipment  is  acquired  for 
test  at  no  cost  to  the  government.  However,  the  supplier  still  owns  the  equipment, 
and  the  government  cannot  test  modifications  or  test  to  extreme  environments  with¬ 
out  the  supplier’s  written  permission.  Also,  the  supplier  has  the  right  to  witness  the 
screening  process;  this  can  present  problems  of  security  in  keeping  multiple  suppliers 
away  from  a  competitor’s  equipment. 

The  alternative  method  is  to  buy  sample  equipments  of  the  leading  candidates. 
This  presents  the  problem  of  funding  the  procurement  if  a  large  number  of  candidates 
or  if  expensive  equipments  are  involved.  If  every  fully  preliminarily  qualified  candi¬ 
date  is  not  included,  the  screen’s  effectiveness  in  gathering  data  for  procurement  is 
defeated.  Tight  specifications  and  accurate  and  complete  records  are  needed  to  keep 
the  number  of  candidates  within  reasonable  limits;  these  same  items  are  required  to 
guard  against  challenges  resulting  from  procurements  based  on  the  screen  results. 
This  disadvantage  is  usually  balanced  by  the  advantages  of  the  government’s  owning 
the  equipment;  early  budgetary  planning  is  needed  to  ensure  adequate  funding  for  the 
test  cycle. 


CONDUCTING  PERFORMANCE  AND 
ENVIRONMENTAL  TESTS 


Screening  tests  should  normally  include  the  following  steps: 

1.  Initial  performance  test  to  the  manufacturer’s  specifications;  where  several 
parameters  interreact,  choose  “worst  case"  test  conditions. 

2.  Testing  to  project  specifications  including  environmental  limits. 

3.  Burn-in  for  100  hours. 

4.  Combined-environment  test  to  specified  environmental  limits  for  tempera¬ 
ture,  humidity,  vibration,  and  electrical  transient  response  as  a  minimum. 
Add  stresses  which  may  be  encountered  in  use  from  environmental  cycling, 
repetitive  shock,  and  EMI  as  appropriate. 

It  is  desirable  to  tost  at  least  three  units  of  each  candidate. 

Normally  these  steps  conclude  the  screening  tests;  the  entire  test  cycle  can  be 
completed  in  as  little  as  3  weeks  for  simple  equipments.  The  combined-environment 
test  (CET)  requires  120  hours.  The  total  test  time  may  be  extended  for  more-complex 
equipments,  by  repair  time  should  failures  occur,  or  by  problems  in  scheduling  the 
test  apparatus. 

The  screening  tests  measure  the  design  suitability  of  the  equipment  in  the 
intended  application;  they  do  not  directly  measure  reliability.  Reliability  predictions 
in  conjunction  with  the  screening  process  are  normally  sufficiently  accurate  for 
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planning  purposes;  however,  reliability  tests  may  be  required  if  a  particular  minimum 
acceptable  reliability  is  essential.  The  cost  of  reliability  testing  can  be  reduced  mark¬ 
edly  by  combining  the  reliability  test  and  the  CET,  using  multiple  equipments  and 
extending  the  test  time  as  necessary  to  accumulate  a  sufficiently  high  equipment  “on" 
time. 


The  Screening  Acceptance  Criteria  are  as  follows: 

1.  No  repeatable  failures  for  any  single  tent  sequence.  (Example:  A  unit 
which  fails  a  vibration  test  twice  is  not  acceptable.) 

2.  No  pattern  failures  (two  or  more  failures  of  the  same  part  in  identical  or 
equivalent  applications  which  are  caused  by  the  same  basic  failure  mecha¬ 
nism). 

3.  Measured  MTBF  meets  or  exceeds  project  requirements  (see  table  XI-1) 
with  an  acceptable  confidence.  High  MTBF  requirements  may  dictate 
using  failure-free  trial  acceptance  criteria  (see  table  XI-2).  All  units  must 
pass  failure-free  within  three  trials  of  the  CET  phase. 

4.  Predicted  performance  remains  within  required  limits  when  weighted  by 
quality  factors.  (See  PREDICTING  PERFORMANCE.) 

EVALUATING  THE  RESULTS 


Ideally,  multiple  candidates  will  be  qualified  by  the  screen  without  reserva¬ 
tion.  The  scrocn  results  can  then  be  used  to  procure  equipments  for  service  use  (see 
chapter  VI).  However,  elements  of  the  intended  application  and  of  the  equipment 
design  generally  do  not  mesh  perfectly,  so  the  screening  decision  is  not  clearly  deter¬ 
mined.  An  economic  analysis  is  normally  required  to  determine  the  acceptability  of 
each  candidate.  By  adapting  the  system  life-cycle  cost  model,  a  sufficiently  accurate 
analysis  can  be  completed.  Subjective  evaluations  will  be  necessary  to  develop  some  of 
the  inputs  to  tho  model;  however,  maintenance  and  logistic  support  data  can  be 
improved  in  quality  by  using  information  supplied  by  current  users  and  obtained 
from  experiences  during  the  test  cycle.  The  data  are  further  fixed  if  warranties  are 
available.  In  general,  reliability/failure  rate  data  are  most  difficult  to  obtain. 

The  reliability  of  established  products  is  already  designed  and  built  in;  talks 
with  the  manufacturer,  observing  his  production  quality  assurance,  and  user  accep¬ 
tance  of  the  product  in  similar  application  environments  combine  to  give  qualitative 
assurance  that  tho  reliability  of  the  equipment  is  adequate.  However,  quantitative 
values  are  roquired  for  further  project  planning.  Reliability  predictions  can  supply 
sufficiently  accurate  values  when  weighted  by  the  screening  data. 

It  is  important  not  to  overlook  the  five  major  issues  which  must  be  resolved 
for  any  acquisition  (see  chapter  VI).  These  qualitative  decisions  may  excludo  one  or 
more  candidates.  The  purpose  of  the  screen  is  to  avoid  development  wherever  possible 
and  economical;  however,  the  satisfaction  of  the  operational  requirements  must  be 
the  paramount  goal  of  the  project. 
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Table  Xf-1.  Table  of  values  for  MTBF  confidence  range  curve. 


Table  XI-2.  Failure-free  acceptance  criteria  for  CET  trials. 


n  =  lot  size 

t  =  CET  test  time  (normally  120  h) 

0  =  specified  MTBF  (must  be  t  x  4  or  greater) 

N  =  number  of  trials  allowed  to  pass  the  entire  lot  (round 
up  to  the  nearest  whole  number) 

(N-n  =  number  of  allowable  failure  trials) 

For  intermediate  ranges  of  0 ,  a  trial  length  can  be  assumed  at  1  CET  cycle  (24 
hours).  For  6  greater  than  1200  hours,  additional  test  time  should  be  added,  if  possi¬ 
ble,  in  24-hour-cycle  increments.  It  is  desirable  for  the  lot  size  to  be  at  least  3  but  less 
than  12.  A  maximum  of  two  failure  trials  is  allowed  for  any  single  equipment. 


PREDICTING  PERFORMANCE 


There  are  three  elements  of  field  performance  which  are  important  to  program 
planning  and  system  implementation: 

Operational  performance 
Field  reliability 
Useful  se_ vice  life 

The  operational  performance  is  essential  because  the  equipment  is  not  worth 
procuring  without  it.  To  predict  operational  performance,  the  figure-of-merit  (FOM) 
discussed  under  PERFORMANCE  PARAMETERS  in  chapter  IV  should  be  utilized. 
Taking  the  values  measured  for  the  critical  performance  factors  and  applying  the 
FOM  model,  a  FOM  can  be  computed  for  each  unit  tested;  this  number  becomes 
FOM(m).  L  is  the  limit  computed  from  the  minimum  specification  parameters  using 
the  FOM  model.  Find  the  expected  service  quantity  in  table  A-2  of  MIL-STD-414  and 
convert  the  inspection  level  code  to  a  sample  size  using  table  B-3.  Using  the  com¬ 
puted  QL  resulting  from  the  table  XI-3A  procedure  and  the  sample  size  corresponding 
to  the  expected  service  quantity,  table  B-5  of  MIL-STD-414  will  give  a  lot  percent 
defective  which  equals  the  percent  risk  of  an  unacceptable  unit  going  into  service  use 
without  further  screening.  Generally,  a  1-percent  risk  of  this  nature  is  very  accept¬ 
able. 


The  most  reliable  method  of  predicting  field  reliability  is  through  reliability 
testing.  MIL-HDBK-781  provides  test  plans  with  various  degrees  of  statistical  accu¬ 
racy  ranging  from  risks  of  10%  to  50%  or  more.  (See  fig.  XI-1.)  However,  the  accuracy 
of  these  predictions  is  determined  by  the  accuracy  of  the  specified  test  environment 
in  simulating  the  field  environment  including  input/output  conditions  caused  by  fail¬ 
ures  in  interfacing  equipment  and  including  all  failure  conditions  during  the  test. 
Even  a  properly  run  test  will  tend  to  overestimate  field  reliability  sig¬ 
nificantly  simply  because  the  tests  measure  MTBF  as  a  design  characteristic  rather 
than  MTBCM,  which  includes  maintenance-induced  failures,  false  removals,  and 
pseudo  failures  and  also  quality-  and  workmanship-related  failures.  Compute  the 
workmanship  factor  in  accordance  with  table  XI-3B.  Compute  the  relationship 
between  “inherent  MTBF*  and  “base  MTBCM’  using  table  XI-3C,  but  temper  the 
results  with  engineering  judgment.  The  predicted  field  reliability  is  the  product  of  the 
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predicted  MTBF  from  the  test  results,  the  quality  level  (from  table  XI-3A),  the  work¬ 
manship  factor,  and  the  MTBCM/MTBF  ratio.  If  test  results  are  not  adequate  to  give 
a  valid  reliability  prediction,  use  the  parts  stress  analysis  prediction  procedure  in 
part  2  of  MIL-HDBK-217  to  establish  a  baseline.  Determine  the  parts  count  predic¬ 
tion  in  accordance  with  part  3  of  MIL-HDBK-217,  and  establish  the  parts  stress 
analysis  prediction  to  parts  count  prediction  ratio  (this  ratio  is  a  measure  of  the 
derating  factor  used  in  the  design).  Multiply  the  stress  analysis  prediction  by  the  pre¬ 
diction  ratio  to  establish  an  approximate  equivalent  to  a  prediction  based  upon  test 
results,  and  predict  field  reliability  as  above. 

The  useful  service  life  of  an  equipment  is  an  important  but  frequently 
neglected  element  of  field  performance.  If  the  useful  service  life  is  exceeded,  the 
operation  and  support  costs  will  rise  markedly,  often  tripling  annually.  There  are 
many  factors  which  affect  service  life  such  js  usage  environment,  design  technology, 
changing  user  requirements,  and  the  time-to-wearout  characteristic.  If  the  equipment 
was  designed  for  a  usage  environment  comparable  to  the  actual  one  and  assuming 
that  the  design  technology  and  user  requirements  do  not  cause  obsolescence,  the 
time-to-wearout  becomes  the  primary  determinant  of  the  useful  service  life. 

The  factors  which  influence  time-to-wearout  include  design  features  and  pro¬ 
duction  quality  and  workmanship.  Many  of  these  factors  are  incorporated  in  the  pre¬ 
diction  of  field  reliability  as  the  time-to-wearout  is  a  function  of  (1)  the  number  of 
repairs  required  and  (2)  the  deterioration  caused  by  the  repair  action.  The  time-to- 
wearout  can,  therefore,  be  expressed  as  a  product  of  the  field  reliability  and  a  repair 
factor.  The  repair  factor  can  be  calculated  in  accordance  with  table  XI-3D;  it  includes 
repair  features,  training  levels,  and  support  system  characteristics  which  are  deter¬ 
mined  by  the  design  of  the  equipment  and  of  the  support  system.  The  results  will  be 
stated  in  hours  and  must  usually  be  converted  to  years  by  dividing  by  the  expected 
usage  (operating  hours  per  y^ar). 

Experience  has  shown  that  there  are  essentially  three  distinct  grades  of  equip¬ 
ment:  commercial,  industrial,  and  military.  Each  grade,  when  used  in  military  appli¬ 
cations.  exhibits  u'eful  service  life  characteristics  as  follows: 

Commercial  1-2  years  and  as  high  as  5  years 

Industrial  5-G  years  and  as  high  as  10  years 

Military  10-12  years  and  as  high  as  30  years 

In  considering  service  life,  cost  and  operational  performance  should  be  taken 
into  account.  An  equipment  with  a  long  expected  service  life  may  not  exhibit  tbe  nec¬ 
essary  performance  to  meet  expected  future  requirements.  A  short-lived  equipment 
may  be  so  inexpensive  in  comparison  to  a  long-lived  equivalent  that  lower  lifecycle 
costs  will  result  from  multiple  procurements.  However,  multiple  procurements  of 
small  quantities  of  peculiar  items  will  not  be  cost-eficc.ivc.  If  the  technology  or  user 
requirement  factors  are  very  unstable,  a  relatively  short  service  life  equipment  may 
be  indicated;  conversely,  an  equipment  inextricably  associated  with  a  ship  or  air  plat¬ 
form  must  serve  for  the  life  of  the  platform. 

Despite  the  deterministic  methods  of  predicting  these  field  performance  fac¬ 
tors,  an  eq  lipment  will  not  begin  to  meet  predictions  without  thorough  planning  and 
execution  of  the  acquisition  and  support  steps.  The  ILS  t  -ks  must  be  completed 
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properly  to  make  the  assumptions  valid  which  underpin  these  prediction  methods.  Of 
the  predictions,  operational  performance  will  tend  to  be  most  accurate  (usually 
2-place  accuracy  is  provided),  and  service  life  will  tend  to  be  least  accurate  (±30%). 
Although  not  precise  in  absolute  terms,  the  predictions  will  tend  to  provide  accurate 
comparisons  between  alternatives.  No  prediction  can  completely  replace  experienced 
engineering  judgment,  buc  the  methods  presented  here  can  be  useful  tools  in  program 
ph  m mg  and  system  implementation. 


Table  XI-8.  Performance  prediction  calculations. 


A. 


B. 


C. 


Quality  Level  Calculation  For  the  i1^  Unit 
X,  =FOM(m)  -  L 

Calculate  the  mean  =  ill —  =  M 
n 


2* 


Calculate  the  variance  = 


Calculate  the  standard  deviation  =  S  Jv 

Calculate  the  quality  level  =  QL  =  tL  (derived  from  MIL-STD-414) 
Workmanship  Factor  Calculation 

D  =  number  of  performance-relevant  mqjor  defects  cited  by  workman¬ 
ship  inspection  in  accordance  with  an  established  standard  such  as 
MIL-STD-252 


N  =  number  of  units  inspected 

P  =  number  of  projected  service  units 

C  =  level  of  complexity  of  unit  inspected  (per  chapter  VI) 

WF  =  J^l-  jf)  or  .i  ,  whichever  is  In  (C  +  1.5)  =  log  2-  •  .82 

(derived  from  MIL-STD-105) 

MTBCM/MTBF  Ratio  Calculation 

Inspect  the  design  for  the  characteristics  listed  and  sum  the  indicated  values. 


XI-11 


Table  XI-S.  Performance  prediction  calculations  (continued). 


Characteristic 

Yes 

No 

Notes 

1.  Failure  alarms 

10 

2 

1,2 

2.  Built-in  diagnostic  test 

20 

1 

1 

3.  Test  points  and  adjustments  are  easily 
accessible  and  protected  from  short  cir¬ 
cuits  to  ground  and  to  surrour.  ling  cir¬ 
cuitry 

30 

3 

i 

Notes:  1.  multiply  by  fraction  of  equipment  functions 

covered 

2.  double  for  redundant  functions 


A  =  sum  of  indicated  characteristic  values 

B  =  fraction  of  functions  with  redundancy  built  in 

MTBCM  _  A_  (i_b)2  (derived  empirically) 

MTBF  60 

D.  Repair  Factor  Calculation 

Evaluate  the  equipment  in  accordance  with  the  values  below  to  establish  the 
maintenance  life  factor. 


1. 

Ease  of  replacement  factor 

plugged  in 

60 

screwed  down 

48 

soldered  (to  terminals) 

40 

soldered  (to  PC  board) 

32 

wire  wrap/crimp/wire  weld 

16 

potted  or  coated 

2 

2. 

Level  of  repair 

Organization 

4 

Intermediate 

8 

Depot 

20 

3. 

Technician  experience  level 

unskilled/E-3  or  below 

2 

semiskilled/E-4  or  E-5 

8 

skilled/E-6  or  above 

16 

Maintenance  life  factor  (MLF)  =  sum  of  values  from  2, 3)  above  —30 

Supply  support  factor  (SSF)  =  ratio  of  preferred  standard  parts  (per  MIL- 
STD-242)  to  total  number  of  parts 

RF  =  1  +  (In  C)(1.82)1-1C  (MLF)  (SSF) 

C  =  complexity  (per  chapter  VI)  (derived  empirically) 
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.2  .4  6  .8  1  2  3  4  5  6  78910  20  40  60  80100  200 

EXPECTED  TEST  TIME  (MULTIPLES  OF  ) 

EXPECTED  COST  =  (EXPECTED  TEST  TIME  (HOURS)  -f-  NUMBER  OF  UNITS  UNDER  TEST) 
(COST/HOUR)  +  (EXPECTED  NUMBER  FAILURES  X  COST/FAILURE) 

Figure  XI-1 .  Risk  as  a  function  of  test  time. 
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XII.  SYSTEM/DEVELOPMENT  SPECIFICATION 


Program-peculiar  system,  development,  and  product  specifications  are 
required  for  system/equipment  items  undergoing  engineering  or  operational  develop¬ 
ment  at  government  expense.  The  top-level  specification  —  the  first  to  be  prepared 
and  the  one  of  concern  to  this  section  —  is  intended  to  establish  the  functional  con¬ 
figuration  identification  of  the  item  being  developed. 

According  to  MIL-STD-490,  there  are  three  types  of  specifications  the  con¬ 
tents  of  which  identify  the  functional  configuration  of  an  item  —  the  Type  A  “system” 
specification  and  the  Types  B1  and  B2  “development”  specifications.  Which  to  use  de¬ 
pends  on  the  complexity  of  the  item.  The  Type  A  specification  is  necessary  for  a  sys¬ 
tem  comprising  “subsystems,  assemblies  (or  sets),  skills,  and  techniques  capable  of 
performing  and/or  supporting  an  operational  (or  nonoperational)  role  to  the  degree  it 
can  be  considered  a  self-sufficient  item  in  its  intended  . . .  environment”  (MIL- 
STD-490).  In  addition  to  requirements  common  to  the  functional  identification  of  all 
hardware  developments,  this  specification  would  require  definition  of  the  extent  to 
which  the  missions  of  the  system  affect  design  requirements,  current  and  potential 
enemy  threats  to  the  system,  operational  and  organizational  concepts  which  con¬ 
strain  design  and  operation,  nuclear  control  requirements,  and  coverage  of  system 
effectiveness  models. 

A  Type  B1  specification  is  applicable  to  a  “prime”  item.  This  item  does  not 
have  the  complexity  of  a  system,  but  in  order  to  define  it  properly  in  the  specification 
it  generally  would  be  complex  enough  to  have  to  include  (1)  functional  flow  diagrams 
to  the  level  required  to  identify  its  several  essential  functions;  (2)  functional  and 
physical  interfaces  between  it  and  other  items  and  between  the  mqjor  components 
within  itself;  (3)  a  rnqjor  component  list;  and  (4)  lists  of  government-furnished  and 
-loaned  property  incorporated  by  the  item.  In  addition,  this  item  will  require  provi¬ 
sioning  actions,  operation  and  maintenance  manuals,  and  quality  conformance 
inspection  of  each  unit  of  the  item. 

The  functional  configuration  of  a  “critical”  item  one  below  the  complexity  of  a 
prime  item  —  can  be  described  adequately  by  a  Type  B2  specification. 

When  a  system  is  covered  by  a  Type  A  specification  and  additional  specifica¬ 
tion  of  its  rngjor  functional  areas  is  necessary  to  completely  identify  system  func¬ 
tional  configuration,  Types  B1  and  B2  specifications  also  are  used  as  second-level  de¬ 
velopment  specifications  to  describe  the  “allocated”  functions  of  subsystems  and  other 
major  items  of  the  system.  The  criteria  of  complexity  described  above  apply  to  which 
Type  B  specifications  to  use  for  allocated  functional-configuration  identification. 

Content  and  format  of  TVpes  A,  Bl,  and  B2  specifications  are  indicated  in 
appendices  I,  II,  and  III,  respectively,  of  MIL-STD-490. 

Besides  the  “type”  of  specification,  which  “form”  to  use  must  be  considered. 
MIL-S-834S0  specifies  the  use  of  four  different  forms  which  differ  in  the  degree  of 
control  they  provide  over  format  and  content.  They  are  identified  as  “Form  la,  Form 
lb,  Form  2,  and  Form  3.”  The  degree  of  control  required  depends  on  how  refined  the 
specification  must  be  to  satisfy  government  needs  for  the  particular  development. 
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Form  la  specifications  are  prenared  to  rigid  military  standards  where  exten¬ 
sive  control  of  paragraph  content  ana  lormat  is  necessary.  They  must  conform  to 
MIL-STD-490  in  all  respects,  including  paragraph  numbering  and  titling  and  subject 
coverage  as  specified  in  the  appendices  of  MIL-STD-490. 

Form  lb  specifications  also  are  prepared  to  military  standards,  but  where  lim¬ 
ited  format  control  is  allowable  along  with  maximum  content  control.  They  can  be 
prepared  according  to  the  requirements  of  MIL-STD-461  or  MIL-STD-490.  If  MIL- 
STD-490  is  followed,  the  strict  paragraph  sequencing,  numbering,  and  titling  of  its 
appendices  are  not  obligatory,  although  the  six-section  format  is  mandatory. 

Form  2  specifications  are  prepared  to  commercial  standards  with  supplemen¬ 
tal  military  requirements  when  such  specifications  will  be  acceptable  for  the  govern¬ 
ment’s  intended  use  (possibly  with  minor  change)  and  offe*  a  price  or  delivery  advan¬ 
tage  over  Form  1  specifications.  The  supplemental  military  requirements  are  (1)  char¬ 
acteristics  must  be  specified  to  the  degree  necessary  to  allow  eventual  procurement 
and  delivery  of  an  item  that  meets  all  government  requirements;  (2)  quality  assurance 
provisions  must  be  included  to  assure  the  meeting  of  these  requirements;  (3)  symbols, 
reference  designations,  codes,  abbreviations,  etc.,  must  be  to  military  standards  or  be 
fully  explained;  and  (4)  the  commercial  standard  to  which  the  specification  is  pre¬ 
pared  must  be  furnished  or  be  referenced  and  available. 

Form  3  specifications  are  prepared  merely  in  accordance  with  the  contractor’s/ 
agency’s  normal  practices,  but  must  satisfy  the  intended  use  of  the  document. 

Although  strict  adherence  to  the  appendices  of  MIL-STD-490  is  not  mandatory 
for  Forms  lb,  2,  and  3  specifications,  use  of  the  appendices  as  a  guide  is  encouraged 
to  assure  adequate  coverage  of  all  requirements  that  may  be  necessary  for  the  par¬ 
ticular  development. 

Types  A,  Bl,  and  B2  specifications  establish  a  functional  configuration 
baseline  against  which  changes  are  evaluated  and  made  as  design  development  pro¬ 
gresses.  When  design  changes  are  approved  and  incorporated,  the  documented 
baseline  is  updated  accordingly  to  ensure  continual  correspondence  between  the 
item’s  actual  configuration  arid  the  documentation  which  describes  it.  This  baseline 
identification  serves  throughout  the  life  cycle  of  the  item  as  a  description  of  required 
functional  characteristics. 

Functional  baselines  should  be  specified  flexibly  to  avoid  undesired,  premature 
commitments  to  detailed  requirements  that  would  be  difficult  and/or  expensive  to 
change.  In  the  case  especially  of  mqjor  system  developments,  the  initial  baseline  iden¬ 
tification  could  well  be  a  preliminary  system  description  of  a  range  of  proposed  broad 
performance  parameters  or  characteristics  which  then  could  be  used  to  facilitate  the 
evaluation  of  alternative  design  approaches  as  performance-cost  tradeoffs  are  made 

A  flexible  functional  baseline  is  a  good  start  toward  later  establishment  of 
product  specifications  which  are  performance  oriented  rather  than  design  oriented  — 
“product  function”  specifications  rather  than  “product  fabrication”  specifications. 
Various  studies  and  current  directives  emphasize  the  preference  of  the  former  over 
the  latter  except  for  developments  where  materials,  individual  parts,  add/or  internal 
configuration  must  be  controlled  because  of  particular  military  needs.  Performance- 
type  specifications  (i.e.,  form,  fit,  function  specifications)  which  include 
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environmental  and  interface  requirements,  as  opposed  to  detailed  design  specifica¬ 
tions  requiring  particular  parts,  processes,  materials,  and  internal  configuration, 
assure  multiple-source  procurement.  Also,  by  including  standardized  mechanical, 
electrical,  and  environmental  interface  requirements  in  performance-type  specifica¬ 
tions  for  items  applicable  to  common  operational  functions,  the  interchangeability  of 
similar  equipments  can  be  ensured.  This  results  in  ready  replacement  of  old  designs 
by  new-generation  equipment  as  well  as  enhancing  design  and  price  competition 
among  contractors.  However,  the  other  side  of  the  coin  —  proliferation  of  detailed 
designs  —  must  be  considered,  too,  for  optimum  end  results.  To  minimize  variety  and 
to  control  design  features  which  pertain  to  interchangeability,  compatibility,  reliabil¬ 
ity,  and  maintainability,  it  may  be  necessary  to  include  some  detailed  design  require¬ 
ments  in  performance-type  specifications. 


In  exploring  and  establishing  system/equipment  requirements  for  specifica¬ 
tion,  cost,  quantity,  and  schedule  constraints  should  be  given  equal  status  to  perform¬ 
ance  and  physical  characteristics.  Cost  factors  should  be  introduced  as  early  in  the 
conceptual  design  and  planning  phases  as  possible,  and  formal  requirements  in  final 
form  should  be  specified  and  issued  only  after  several  iterations  of  cost-performance 
estimates.  Not  only  should  first  cost  (design  to  cost)  be  weighed  carefully,  the  costs  of 
maintenance,  supply,  and  training  —  along  with  the  tradeoff  values  of  cost,  reliabil¬ 
ity,  performance,  maintainability,  and  efficiency  —  should  be  carefully  considered. 

The  following  features  can  increase  cost: 

Needless  refinement 

Expensive  finishes 

Unreasonable  tolerances 

Critical  materials 

Restrictive  processes 

Overemphasis  on  appearance 

Overlong  life  expectancy  as  related  to  intended  use 

Overprotcctian  against  failure 

Unreasonable  ana  excessive  inspection 

Unrealistic  reliability 


A  special  case  exists  when  establishing  requirements  to  be  specified  for  items 
which  are  to  be  supported  initially  by  contractor  long-term  warranties.  Here,  reliabil¬ 
ity,  maintainability,  and  initial  provisioning  are  not  the  mqjor  concerns  they  other¬ 
wise  would  be.  These  requirements  may  be  relaxed  along  with  their  quality  assurance 
provisions  when  there  are  adequate  contractual  warranty  provisions  which  make  it 
profitable  for  the  contractor  to  design  and  build  highly  reliable  and  maintainable 
equipment. 


Regardless  of  the  scope  of  requirements  in  a  specification,  each  requirement 
should  be  stated  to  present  exactly  what  the  government  wants.  Requirements  should 
be  so  worded  as  to  provide  a  definite  basis  for  rejection  when  testing  and  inspection 
reveal  unsuitability.  Also,  care  should  be  exercised  to  avoid  unrealistic  requirements 
and  those  which  conflict  with  referenced  documents.  If  requirements  are  not  abso¬ 
lutely  clear  and  definite,  bidders  will  not  know  exactly  what  they  are  to  furnish  in 
order  to  make  a  responsible  bid. 


XI 1-3 


The  contents  of  a  specification  pertain  directly  and  only  to  the  item  to  be 
developed.  Thus,  the  following  activities  of  a  development  program  are  not  covered  by 
a  specification,  but  rather  by  the  statement  of  work  of  the  contract: 

Configuration  management 

Integrated  logistics  support 

Safety  program 

Human  factors  program 

Training  program 

Level  of  repair 

Acquisition  management 

Supply  support 

Contractor  services 

Quality  program 

Reliability  program 

Maintainability  program 

Planned  maintenance  subsystem 

Test  point  program 

Calibration  and  instrumentation 

Electromagnetic  compatibility  program 

Special  test  programs 

In  addition,  the  specification  must  not  contain  requirements  for  the  delivery 
of  data.  Data  to  be  delivered  under  the  contract  can  only  be  ordered  by  the  Contract 
Data  Requirements  List,  DD  Form  1423.  The  form  and  content  of  each  line  item  of 
data  on  the  DD  Form  1423  are  required  to  be  specified  by  a  Data  Item  Description, 
DD  Form  1GG4,  which  is  attached  to  and  becomes  a  part  of  the  DD  Form  1423.  When 
data  requirements  are  contained  in  a  DD  Form  1GG4,  such  requirements  must  not 
appear  in  the  specification  except  that  deliverable  data  can  be  identified  in  Section  G, 
Notes,  of  the  specification.  This  identification  should  include  the  corresponding  DD 
Form  1GG4  numbers.  An  exception  allowing  data  requirements  to  be  described 
directly  in  a  specification  occurs  when  it  is  impractical  to  separate  such  description 
from  its  context.  An  example  of  this  would  be  data  requirements  covering  quality 
assurance  provisions. 
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Life-Cycle  Cost  Analysis  for  Detail  Design ...  XIII-3 
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XIII.  DECISION  TO  BUILD,  BUY,  OR  MODIFY 


Commercial  enterprises  base  their  make-or-buy  decisions  on  economics. 
The  military  frequently  bases  the  decision  on  expediency.  The  design 
and  procurement  approach  should  result  in  the  provision  of  the 
required  equipment  characteristics  to  the  Navy  at  lowest  system  life 
costs.  This  1;  done  through  the  use  of  life-cycle  cost  (LCC)  analyses,  as 
follows. 


SELECTION  OF  THE  BEST  CANDIDATE 
EQUIPMENT/DESIGN 


Identify  all  candidate  equipments  —  existing  and  to  be  developed  —  and  their 
requirements  for  operation  and  maintenance  during  the  deployment  phase  of  the  sys¬ 
tem  life  cyclo.  Eliminate  those  that  cannot  meet  minimum  requirements  for  the  new 
mission.  On  the  basis  of  total  life  cost  analyses,  the  equipment  having  the  lowest 
total  life  cost  design  which  provides  the  minimum  essential  performance  should  be 
selected  for  procurement.  In  addition  to  total  life  costs,  quality,  delivery,  acquisition 
costs,  and  political  factors  must  be  considered  (although  total  life  cost  should  pre¬ 
dominate)  so  that  the  greatest  project  effectiveness  is  achieved. 

MILITARY  OFF-THE-SHELF  EQUIPMENT 


Determine  whether  there  are  any  off-the-shelf  military  (other  service  ns  well 
as  Navy)  equipments  that  could  provide  the  require  characteristics  without  modifica¬ 
tion  or  after  being  modified.  Obtain  the  existing  procurement  specifications  of  these 
equipments  and  one  of  the  units  of  each  to  review  their  characteristics.  Remember 
that  even  existing  military  equipment  may  require  new  interfaces  for  the  new  mis¬ 
sion.  Chock  with  the  cognizant  acquisition  agency  to  find  out  what  deviations  and 
waivers  have  been  granted  against  the  specifications. 


FEASIBILITY  OF  MODIFYING  MILITARY  EQUIPMENT 


The  criteria  for  deciding  whether  or  not  modifications  to  military  off-the-shelf 
equipments  are  feasible  are: 

•  Will  meet  new  requirements 

•  Performance  will  not  be  degraded 

•  Service  life  will  not  be  decreased 

•  Availability  (reliability  and  maintainability)  will  not  be  decreased 

•  Will  be  cost-effective 
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COMMERCIAL  OFF-THE-SHELF  EQUIPMENT 


Determine  if  there  are  any  off-the-shelf  commercial  equipments  that  could  pro¬ 
vide  the  required  characteristics,  without  modifications  or  after  modification.  See 
chapter  XI,  Screening  Techniques,  for  details.  Even  without  modification,  new  inter¬ 
faces  may  be  required  for  the  military  mission. 

NEW  DESIGNS 

Since  the  costs  during  the  deployment  phase  (assumes  a  quantity  greater  than 
10)  always  exceed  development  and  procurement  costs,  consider  new  design  concepts 
that,  compared  to  the  candidate  off-the-shelf  equipments,  are  aimed  at  less  expensive 
operation,  maintenance,  and  support.  Develop  or  expand  the  new  design(s)  to  the 
extent  necessary  to  provide  the  input  data  required  by  comparative  LCC  analysis  (see 
chapter  IX,  Life-Cycle  Costing).  Be  sure  now  design  and  utilization  concepts  include  a 
well  planned  maintenance  concept  (see  MAINTAINABILITY  PROGRAM  in  Chapter 
X)  and  the  use  of  proved  technology  (no  forcing  the  state  of  the  art)  and  of  compo¬ 
nents  commonly  used  in  industry. 

New  designs  are  often  preferred  in  order  to  obtain  the  advantage  of  m-house 
control  which  allows  the  program  manager  to  influence  the  design.  Such  control  and 
influence  are  not  present  for  existing  equipments,  though  modifications  permit  some 
opportunity  for  reconfiguring  an  equipment  to  the  latest  desires  as  well  as  require¬ 
ments. 


The  problems  associated  with  new  designs  involve  unknown  cost,  unknown 
development  time,  and  the  risk  that  the  new  development  will  not  perform  as  speci¬ 
fied  Any  two  of  these  three  factors  may  be  defined  and  controlled,  but  the  third  one 
will  always  be  an  unknown.  Some  of  the  elements  that  comprise  the  development 
costs  are  labor,  management,  documentation  (administrative,  procurement,  engineer¬ 
ing,  and  support),  inventory  (procurement,  storage,  and  control),  fabrication,  quality 
assurance,  and  testing  (performance,  environmental,  reliability,  maintainability, 
safety,  etc.).  There  is  also  a  learning  curve,  wherein  the  first  units  always  cost  more 
than  following  identical  units  on  the  same  production  line. 


DEVELOPMENT  OF  MODIFICATIONS 


For  military  or  commercial  off-the-shelf  equipment  requiring  modification,  the 
modification(x)  will  require  developmental  effort.  The  selected  equipment  should  be 
developed  to  the  extent  that  a  development  specification  (Type  B,  MIL-STD-490) 
could  be  prepared  in  sufficient  detail  to  effectively  describe  the  characteristics  which 
must  be  finalized  through  engineering  development  into  a  production  model.  That  is, 
the  equipment  must  be  developed  sufficiently  at  this  point  so  that  the  requirements 
for  its  engineering  development  can  be  specified  well  enough  to  get  this  phase  of  the 
effort  off  to  a  good  start  and  end  in  a  cost-effective  production  design  and  specifica¬ 
tion  Check  chapter  XVI,  Types  of  Contracts,  and  chapter  XV,  Warranty  Applications 
Development  of  modifications  should  include  considerations  similar  to  those  dis¬ 
cussed  for  new  designs  in  the  next  paragraph. 
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DEVELOPMENT  OF  NEW  DESIGNS 


In  new  designs,  all  equipment  characteristics  have  to  be  addressed  in  a  devel¬ 
opment  effort.  Refer  to  chapter  III,  Program  Planning.  The  extent  and  detail  of 
design  delineation  required  at  this  point  for  a  new  design  can  be  inferred  by  reference 
to  MIL-STD-490,  Appendixes  II  and  III,  which  define  the  content  requirements  of 
Type  B  development  specifications. 

LIFE-CYCLE  COST  ANALYSIS  FOR  DETAIL  DESIGN 


For  modification  or  new  designs,  urgent  consideration  should  be  given  to  the 
use  of  LCC  analysis,  commensurate  in  scope  and  expense  to  the  overall  project,  for 
helping  to  establish  the  design  details  that  must  be  delineated.  The  LCC  model  of  the 
selected  design  which  was  used  for  the  comparative  analysis  can  be  amplified  to  be  of 
value  at  this  point.  It  then  can  be  updated  and  revised  as  it  is  used  to  help  determine 
design  refinements  and  approved  engineering  changes  which  occur  during  engineer¬ 
ing  development.  The  LCC  analyst  must  be  provided  the  necessary  additional  input 
variables  relating  to  equipment  characteristics  (see  chapter  IX,  Life-Cycle  Costing, 
for  use  in  this  LCC  modeling). 


Table  XUl-1.  Summary  of  the  Buy,  Build,  Modify  Decision 


The  Buy  Alternative  (“Procure," 
rX£.oadcY£logjncntaI  Item  NDD"  - 

ADVANTAGES 

•  Provides  item  quickly 

•  Avoids  development  $ 

•  Shares  broader  production  base 
with  other  applications 


DISADVANTAGES 

•  Item  design  may  not  be  for  sim¬ 
ilar  usage  environment 

•  If  commercial  item  —  may  pre¬ 
clude  low  levels  of  design  own¬ 
ership,  standardization  or 
repair 

•  If  previously  qualified  for  MIL 
environment  —  check  waivers 
granted 

USAGE 

•  Almost  always  best  for  non  vital 
applications 

•  Consider  first 


The  Modify  Alternative  The  Build  Alternative  (“Develop") 


ADVANTAGES 

•  Can  achieve  the  ad-  • 

vantages  of  both 
“builtr  and  “buy" 
when  suitable  mod-  • 
ifiable  item  exists 

•  Can  address  unique  • 

MIL  requirements 

DISADVANTAGES 

•  Can  suffer  all  disad-  • 
vantages  of  both 
“buikr  and  “buy"  if 

-  improperly  designed  • 

-  or  poorly  managed 

-  or  suitable  item  to  • 
modify  does  not 
really  exist 

-  or  information  for 
modification  design 
cannot  be  obtained 

USAGE 

•  Consider  modifica-  • 

tion  of  ideal  system 
partitioning  to  min¬ 
imize  new  design  • 

and  to  keep  new 
design  in  **one  place" 

•  Consider  second 


ADVANTAGES 

Provides  suitable  items  for 
unique  requirements 

Design  can  be  tailored  for  the 
lowr*  support  costs 

Can  usually  achieve  more  capa¬ 
bility  in  smaller,  lighter,  package 

DISADVANTAGES 

Item  usually  small  quantity  pro¬ 
curement 

Development  $  &  time 

More  likely  to  generate  peculiar 
support  requirements 


USAGE 

Most  often  suitable  for  vital 
requirements 

Consider  as  last  resort 
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XIV.  CONSIDERATION  OF  STANDARDIZATION 


Standardization  is  a  highly  controversial  subject.  The  essence  of  this  section  is 
that  DoD  requires  standardization  because  of  its  many  benefits.  Standardization  also 
has  many  disadvantages.  Fortunately,  there  are  various  degrees  of  standardization, 
and  it  is  wise  to  select  the  appropriate  degree  of  standardization  for  the  particular 
program. 

The  Navy  has  committed  itself  strongly  to  a  standardization  program  as  evi¬ 
denced  by  the  various  directives,  instructions,  standards,  and  specifications  on  the 
subject.  J.S.  Gansler,  Assistant  Director  in  the  office  of  Defense  Research  and  Engi¬ 
neering,  stated  that  standardization  “is  a  mqjor  step  in  our  cost-reliability  attack” 
and  that  “standardization  can  and  must  be  achieved"  (June  1975).  In  essence  what 
the  proponents  of  standardization  are  saying  is  that  they  do  not  want  to  repeatedly 
pay  for  developing  the  same  equipment.  A  highly  informative  review  of  this  subject  is 
to  be  found  in  an  Electronics  “X”  project  report  by  ARINC.  For  instance,  “The  reli¬ 
abilities  achievable  for  SHP  (Standard  Hardware  Program)  modules  are  no  greater  or 
less  than  for  other  modules  of  similar  complexity  and  technology  subjected  to  the 
same  quality-control  provisions.”  Also,  “Among  those  who  contributed  information 
...  it  was  the  consensus  that,  to  date,  increased  costs  are  involved  in  the  develop¬ 
ment  of  systems  utilizing  SHP  concepts.” 

Those  within  industry  who  were  questioned  on  this  subject  for  the  TELCAM 
task  indicated  a  dislike  of  the  idea  of  a  requirement  to  use  standard  modules.  Telling 
the  manufacturer  how  to  design,  fabricate,  and  install  essentially  relieves  him  of 
responsibility  for  the  result.  This  in  turn  destroys  incentive.  Actually,  industry  has 
standardized  many  components,  procedures,  and  designs  through  their  joint  efforts  in 
technical  organizations.  Acceptance  of  these  standards  is  voluntary,  but,  as  the  Elec¬ 
tronics  “X”  report  states,  “. . .  good  standards  do  not  have  to  be  ‘enforced.’  They  are 
accepted  because  they  make  economical  and  technical  sense  to  all  concerned.”  Fur¬ 
ther  standardization  by  industxy  is  accomplished  by  its  acceptance  of  circuit  designs, 
modules,  components,  etc.,  due  to  their  usage  history  within  the  industry.  This  stan¬ 
dardization  may  not  always  be  formalized  by  the  term  “standard,”  but  the  results  are 
the  samo.  These  items  are  repeatedly  used  because  their  history  of  reliability,  cost- 
effoctivencss,  replaceability,  and  safety  contributes  to  a  desirable  and  competitive 
product. 

Many  of  those  interviewed  from  within  industry  felt  that  the  required  use  of 
SHP  modules  stifled  their  engineers’  creativity  and  meant  the  expenditure  of  time 
and  money  to  “design  in”  tho  standardized  modules.  Industry  does  make  use  of  pre¬ 
vailing  commercial  standard  components  and  parts  where  suitable  for  their  product; 
but  where  and  when  to  use  them  is  the  individual  decision  for  each  firm.  For  in¬ 
stance,  some  firms  such  as  Motorola,  Scottsdale;  stock  nothing  but  high-reliability 
parts  such  as  JAN-TX.  Because  of  the  volume  of  parts  purchased  they  can  do  this  at  a 
price  equivalent  to  lower-reliability  parts  purchased  on  fragmented  buys.  These  parts 
are  used  for  design  work  as  well  as  production. 

The  military  achieves  a  form  of  standardization  by  repeatedly  procuring  iden¬ 
tical  equipments  over  a  period  of  years.  This  results  in  technological  obsolescence  and 
mediocre  reliability. 
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At  the  same  time,  other  similar  equipment  is  procured  for  slight  variations  in 
mission  requirements  or  improved  equipment  capabilities  for  the  original  require¬ 
ments,  or  because  new  technology  surpasses  some  facet  of  original  equipment  technol¬ 
ogy.  This  results  in  an  excessive  proliferation  of  alternative  equipments  and  defeats 
any  benefits  of  repeatedly  procuring  identical  equipments. 

Where  only  minimal  organizational  maintenance  is  required,  a  better 
approach  might  be  interface  standardization  at  the  level  of  the  black  box,  line 
replaceable  unit,  or  weapons  replaceable  (repairable)  assembly.  This  concept  is  widely 
used  in  the  Navy,  especially  in  missiles  and  avionics. 

The  airline  industry  has  been  using  interface  standardization  for  years.  Com¬ 
bined  with  functional  specifications,  they  call  it  “form,  fit,  function”  standardization. 
The  internal  configuration  is  free  to  evolve  as  technology  changes,  taking  advantage 
of  new  techniques,  devices,  and  materials.  Organizational  maintenance  consists 
mainly  of  replacing  black  boxes  for  repair  under  warranty  or  at  repair  depots. 

The  military  can  adopt  this  plan  and  standardize  the  functions,  interfaces, 
components,  and  workmanship.  Since  internal  circuits  and  configuration  are  not 
specified  or  standardized,  the  manufacturer  is  free  to  give  his  best  effort  in  the  inter¬ 
nal  design  Competition  breeds  innovation  and  reduced  prices.  The  militaiy  will  end 
up  with  the  functional  capability  specified,  the  optimum  technology  at  the  time  of 
purchase,  and  the  lowest  price.  Later  models  of  the  same  functional  item  will  be  dif¬ 
ferent  internally,  but  will  be  fully  interchangeable  with  the  initial  equipment.  The 
need  for  modifications  to  an  installation  to  accommodate  the  now  equipment  is  elimi¬ 
nated. 


Organizational  maintenance  will  not  extend  below  the  modular  level  or  per¬ 
haps  the  black  box  level,  depending  on  mission  requirements  and  the  logistics  pro¬ 
gram  involved.  The  maintenance  concept  will  include  a  suitable  combination  of 
throwaway  modules,  warranties,  and  depot  maintenance.  See  chapter  X,  Integrated 
Logistics  Support,  for  more  details. 

General  worst-case  standards  have  been  developed  for  various  military  envi¬ 
ronments.  Equipment  developed  to  withstand  such  environments  is  expensive.  Where 
it  is  known  that  the  actual  environment  for  an  equipment  will  be  less  severe  than 
“standard,”  it  will  mean  savings  to  modify  the  requirements  to  the  actual  environ¬ 
ment.  This  is  permitted  by  MIL-E-16400  and  various  environmental  standards  includ¬ 
ing  MIL-STD-810,  MIL-STD-167,  MIL-STD-108,  and  MIL-STD-469. 

A  well-organized  standardization  program  takes  advantage  of  standardiza¬ 
tion’s  benefits  and  avoids  its  pitfalls.  To  accomplish  this,  the  project  manager  must 
be  aware  of  the  advantages  and  disadvantages. 
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POTENTIAL  ADVANTAGES  OF 
STANDARDIZATION 


1.  Standardization  reduces  life-cycle  cost. 

2.  It  makes  practical  a  larger  design  budget  due  to  the  larger  number  of  units 
for  amortization  of  design  cost. 

3.  The  number  of  types  of  parts  to  be  purchased  and  stocked  is  reduced. 

4.  Standardization  reduces  the  unit  cost  of  necessary  repair  parts  by  requir¬ 
ing  larger  quantities  of  relatively  fewer  types. 

5.  It  increases  the  quantity  of  the  same  item,  which  reduces  its  unit  cost 
through  economies  of  scale,  mechanized  processes,  and  longer  production 
runs  of  fewer  designs. 

6.  Larger-quantity  productions  make  practical  the  use  of  LSI  integrated  cir¬ 
cuits. 

7.  The  number  of  odd  or  unusual  items  is  reduced. 

8  Production  cost  is  reduced  in  terms  of  tooling  duo  to  higher  usage  of  fewer 
setups. 

9.  Fewer  types  of  test  equipment  are  required. 

10  Simplified  test  procedures  and  fault  isolation  become  possible.  Automated 
test  equipment  becomes  more  practical  to  design  and  produce. 

1 1.  Training  requirements  and  skill  levels  required  for  maintent.  ice  are 
reduced. 

12.  Standardization  increases  maintainability  and  minimizes  equipment  down¬ 
time  due  to  common  spares  and  off-line  repair  or  throwaway. 

13.  The  number  of  different  documents  is  reduced. 

14.  Predictability  is  ine'eased  for  reliability,  maintainability,  safety,  costs,  etc 

15.  Reliability  is  increased  through  evolutionary  redesign. 

16  Availability  is  increased  due  to  increased  reliability  and  maintainability. 

17.  Universal  packaging  becomes  possible  through  standardization. 

18.  Uniformity  in  size,  shape.  ;nd/or  connectors  simplifies  storage  and  testing 
requirements. 

19.  Standardization  permits  parts  or  modules  to  be  interchanged  between  or 
within  systems  and  equipments. 

20.  It  provides  modular  ‘building  blocks'  for  electronic  and  mechanical  design 
(which  speeds  design  and  reduces  development  time). 
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21.  Time  and  expense  are  saved  in  determining  optimum  enclosure  size,  mod¬ 
ule  size,  circuit  board  size,  etc. 

22.  Standardization  reduces  redundant  design  efforts,  reinventing  the  wheel, 
or  redeveloping  existing  equipment. 

23.  It  permits  universal  use  of  carefully  designed  solutions  to  problems  such  as 
cooling,  electromagnetic  interference,  shock,  and  vibration,  thus  reducing 
design  time  and  documentation  requirements  while  improving  the  design 
integrity. 

24.  The  number  of  similar  equipments  is  reduced  along  with  their  development 
time,  costs,  and  documentation. 

25.  Multiple  use  or  broader  application  of  an  item  becomes  possible. 

26.  The  cost  of  adding  a  nonstandard  item  to  the  stock  system  is  eliminated. 

27.  Instructive  guidelines  are  provided  to  the  uninitiated,  which  allows  “nov¬ 
ices”  to  compete  for  the  development  and  production  of  military  electronic 
equipment. 

28.  Standardization  program  quality  procedures  allow  higher  quality,  high- 
reliability  items  at  a  lower  cost  of  implementation.  The  end  products  are 
more  uniform  and  predictable. 

There  are  always  some  things  we  would  like  to  see  done,  but  the  question  is 
how  much  we  are  willing  to  pay  for  it  versus  how  much  we  will  actually  benefit  from 
it.  Military  standards  and  specifications  tend  to  be  excessively  long  and  complex,  slow 
to  follow  the  rapid  advances  in  technology,  and  difficult  and  expensive  to  implement 
They  often  fail  to  produce  the  intended  results.  Numerous  military  standardization 
programs  have  been  tri"d.  Most  have  died  through  lack  of  use.  The  causes  for  such 
lack  of  use  are  among  the  following  disadvantages  of  standardization. 


TYPICAL  DISADVANTAGES  OF 
STANDARDIZATION 


1.  Standardization  “benefits'  are  often  ideals  that  cannot  be  realized. 

2.  Standards  are  usually  prepared  by  designers  of  standards  rather  than 
designers  of  equipment. 

3.  High  initial  costs  are  inherent  in  standardization  programs. 

4.  Components  which  are  larger,  more  expensive,  or  unnecessary  except  for 
standardization  purposes  add  to  equipment  and  repair  costs 

5.  Repair  by  replacement  of  expensive  subassemblies  is  not  cost-effective  in 
cases  where  it  is  obvious  that  an  individual  component  has  failed.  Stan¬ 
dardization  at  the  modular  level  can  lead  to  difficulty  in  locating  what 
would  otherwise  be  a  simple,  low-cost  repair  of  an  obvious  failure. 
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6.  Standardization  limits  design  flexibility  and  alternatives  (which  increases 
cost). 

7.  Standardization  cannot  satisfy  all  requirements  (can’t  please  everyone). 

8.  Standardization  tends  to  have  a  narrow  view  and  does  not  always  consider 
all  parameters  as  variables,  but  rather  fixes  certain  ones  as  constants.  This 
results  in  unreasonable  constraints  to  the  designer  and  possibly  unreliable 
equipment  in  service. 

9.  It  usually  involves  less  than  optimum  electrical  and  mechanical  design 
parameters.  “Optimum'  for  one  circuit  system,  etc.,  is  not  optimum  for  the 
next. 

10.  Generally  inefficient  systems  result  from  standardization.  Problems 
include  excessive  weight,  excessive  volume,  excessive  heat,  and  excessive 
complexity.  Standards  designed  for  expected  worst-case  conditions  produce 
inherently  inefficient  results,  and  cost  more  than  optimum  items. 

11.  Equipment  standardized  internally  has  low  volumetric  efficiency  due  to 
additional  required  connectors  and  hardware,  and  larger  than  optimum 
(but  standard)  connectors,  hardware,  and  components,  all  assembled  m  a 
larger  than  optimum  (but  standard)  module,  subassembly,  case,  cabinet, 
etc. 

12.  Standardization  generally  conflicts  with  “best  commercial  practices.’ 

13.  Equipment  designed  to  a  multitude  of  standards  to  fit  a  variety  of  needs 
generally  suits  none  of  the  needs  well.  Duo  to  extensive  tradeoffs  and  com¬ 
promises,  tho  result  may  be  unsuitable  for  any  of  the  intended  applica¬ 
tions. 

14.  Standard  modules,  circuits,  etc.,  are  frequently  peculiar  to  the  one  system 
for  which  they  wore  originally  designed,  and  cannot  be  designed  into 
another  system  without  causing  deficiencies,  inefficiency,  or  unnecessary 
complexity. 

15.  Items  designed  and  built  to  general  standards  have  trouble  adapting  to 
special  requirements. 

16  There  can  be  otherwise  unnecessary  and  cumbersome  interface  restrictions 
required  for  the  sake  of  standardization;  e.g.,  connectors,  voltage  levels, 
and  module  sizes. 

17.  An  increase  in  the  number  of  connections  leads  to  reduced  reliability. 

18.  Increased  conductor  lengths  may  cause  cross  talk  or  noise,  or  other  EMI 
problems. 

19.  Long  lead  times  (often  due  to  low  usage)  are  common  for  standard  items. 

20.  Commercial  managers  may  be  unwilling  to  change  to  military  standards 
from  their  in-house  standards  for  psychological,  economic,  or  sound  techni¬ 
cal  reasons. 
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21.  Standardization  can  add  to  the  complexity  of  fault  isolation. 

22.  When  a  defective  subassembly  is  isolated,  a  modular  replacement  philoso¬ 
phy  may  preclude  module  repair  due  to  lack  of  detailed  internal  informa¬ 
tion  or  lack  of  repair  parts. 

23.  Standardization  requires  that  adequate  documentation  regarding  the  stan¬ 
dard  be  made  available  to  users. 

24.  It  tends  to  be  arbitrary  for  the  sake  of  documentation. 

25.  Sometimes  “the  tail  wags  the  dog”;  standardization  controls  and  directs 
the  program  rather  than  being  a  tool  of  the  project  manager. 

26.  Standardization  is  sometimes  more  politically  than  economically  or  techni¬ 
cally  motivated. 

27.  Standards  become  obsolete  even  if  valid  when  written  (often  obsolete 
before  published).  This  limits  improvements  in  performance,  cost,  size, 
weight,  or  reliability.  A  good  (rather  than  arbitrary)  standardization  pro¬ 
gram  requires  time  to  develop.  Unfortunately,  this  sometimes  leads  to  a 
“new  standard"  which  becomes  obsolete  before  it  is  finalized. 

28.  Standardization  programs  generally  do  not  last.  Needs,  missions,  and  state 
of  the  art  are  all  constantly  changing.  Standardization  is  an  attempt  to 
hold  on  to  the  present  and  assumes  that  the  particular  standard  will 
remain  useful  for  an  indefinite  period  of  time. 

In  theory,  the  advantages  of  standardization  clearly  outweigh  the  disadvan¬ 
tages.  In  the  real  world,  however,  “standardization”  is  ofton  no  more  than  a  manage¬ 
ment  technique  to  maintain  control  over  an  otherwise  constant  stream  of  changes. 
The  changes  may  be  improvements,  but  if  designers  continue  to  improve,  nothing 
ever  gets  built. 

Excessive  or  mismanaged  standardization  can  be  bad  and  can  become  very 
expensive  and  yield  obsolete  systems  with  excessive  size,  weight,  etc. 

There  is  always  some  natural  standardization  in  any  project.  The  questions 
are  how  much  and  whether  it  should  be  enforced  If  enforced,  how  much  enforcement 
and  by  what  means.  Allow  flexibility  in  the  application  of  specifications  and  stan¬ 
dards  to  increase  cost-effectiveness.  Encourage  standardization  as  a  means  of  life¬ 
cycle  cost  reduction,  but  remember  that  excessive  standardization  cm.  lead  to  exces¬ 
sive  complexity  and  excessive  cost. 

In  order  to  develop  standards,  many  variables  that  are  extreme  for  one  pur¬ 
pose  must  be  made  extreme  for  all  uses  (which  drives  up  the  cost  of  the  system),  or 
else  must  be  made  less  extreme  to  reach  a  compromise  between  the  various  require¬ 
ments  (which  may  make  the  system  unsuitable  for  any  purpose).  There  is  a  third 
alternative,  to  develop  a  standard  with  specified  options  or  defined  classes  of  use  to 
meet  the  independent  needs  of  different  users’. 

Another  use  of  standards  is  to  put  on  paper  constantly  changing  interface 
requirements.  This  ensures  that  two  or  more  groups  designing  portions  of  a  system 
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will  have  a  common  meeting  ground.  Only  interlace  standards  (form,  fit,  and  func¬ 
tion)  are  required  for  this. 


DEFENSE  STANDARDIZATION  PROGRAM  (DSP) 


The  Navy's  role  in  the  Defense  Standardization  Program  is  outlined  in  SEC- 
NAVINST  4120.3E.  An  enclosure  to  the  instruction  in  DoD  Directive  4120  3,  which 
defines  standardization  as  the  adoption  and  use  (by  consensus  or  decision)  of  engi¬ 
neering  criteria  to  achieve  the  following  objectives: 

1.  To  improve  the  operational  readiness  of  the  military  services  by  increasing 
efficiency  of  design,  development,  material  acquisition,  and  logistics  sup¬ 
port. 

2.  To  conserve  money,  manpower,  time,  facilities,  and  natural  resources. 

3.  To  minimize  the  variety  of  items,  processes,  and  practices  which  are  asso¬ 
ciated  with  design,  development,  production,  and  logistics  support  of 
equipments  and  supplies 

4.  To  enhance  interchangeability,  reliability,  and  maintainability  of  military 
equipments  and  supplies. 

DSP  PLANS  FOR  ACHIEVEMENT 

Tho  Defense  Standardization  Program  seeks  to  achieve  these  objectives 
through: 

1.  Management  and  engineering  actions  required  to  establish  and  effectively 
implement  standardization  agreements  and  decisions. 

2.  Establishing  and  maintaining  uniform  and  teclmically  adequate  records  of 
tho  engineering  definition  of  equipments  and  supplies. 

3.  Promoting,  in  support  of  procurement,  maintenance,  supply,  and  future 
design,  the  reuse  of  records  of  engineering  definitions,  the  engineering  cri¬ 
teria  therein,  and  previously  developed  or  acquire  material  represented  by 
these  records. 

4.  Prescribing,  for  specifications,  standards,  drawings,  and  other  standardiza¬ 
tion  association  documentation,  (a)  format,  (b)  procedure  for  effective  coor¬ 
dination,  (c)  quality  of  documentation,  and  (d)  procedures  for  collating  and 
disseminating  this  information. 

POLICIES  OF  THE  DSP 

1.  Military  Operational  Reouirements.  Wherever  feasible,  military  opera¬ 
tional  requirements  for  material  shall  be  satisfied  through  the  use  of  exist¬ 
ing  military  designs  or  commercial  products.  If  military  need  can  be 
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satisfied  only  through  new  development,  the  new  development  authorized 
shall  encompass,  to  the  greatest  extent  practicable,  all  equivalent  needs  of 
the  Military  Departments  and  Defense  Agencies. 

2.  Exploratory  Development  and  Advanced  Development.  The  categories  of 
Exploratory  Development  and  Advanced  Development  represent  scientific, 
experimental,  and  early  development  efforts  aimed  at  innovation  and 
evaluation  of  feasibility.  The  use  of  existing  standard  items  and  engineer¬ 
ing  practices  is  advocated  in  the  interests  of  economy,  where  these  satisfy 
the  needs  of  such  program  efforts.  However,  use  of  standards  shall  be  sec¬ 
ondary  to  the  prime  objective  of  these  development  categories;  e.g.,  proof 
of  a  concept. 

3.  Engineering  Development  and  Operational  System  Development  In  the 
categories  of  Engineering  Development  and  Operational  System  Develop¬ 
ment  where  the  systems  and  equipment  are  engineered  for  eventual  Serv¬ 
ice  use,  a  maximum  degree  of  standardization  shall  be  achieved  without 
causing  unacceptable  compromise  of  performance,  reliability,  timely  avail¬ 
ability,  or  cost  of  systems  and  without  preventing  the  applications  of  the 
most  advanced  proved  techniques  or  hardware. 

4.  Procurement.  Techniques  that  provide  opportunities  to  achieve  standardi¬ 
zation  objectives  during  procurement  include  (a)  the  specification  of  com¬ 
plete  design  requirements,  (b)  multiyear  procurement,  and  (c)  negv.  .iation 
authorized  by  Paragraph  3-213  of  Armed  Services  Procurement  Regulation 
to  achieve  standardization  of  technical  equipment. 

5.  Snpnlv.  The  variety  of  types,  kinds,  and  sizes  of  items  of  supply  shall  be 
reduced  to  the  minimum  consistent  with  effective  support  of  military  op¬ 
erations. 

6.  Standardization  Planning.  Standardization  efforts  will  be  planned  and 
managed  with  a  view  to  establishing  (a)  timely  design  standards  that 
reflect  current  technology,  and  (b)  standards,  specifications,  and  other 
documentation  that  offer  the  greatest  advantages  for  cost  reduction,  for 
item  reduction,  for  competitive  procurement,  for  simplification  of  mainte¬ 
nance  and  logistics  support,  for  increased  reliability,  or  for  increased 
design  efficiency. 

7.  Management  of  Engineering  Information. 

a.  Organization  of  Engineering  Information.  Engineering  information, 
obtained  from  design  and  development  and  of  the  type  reasonably 
expected  to  be  necessary  for  future  reuse,  shall  be  organized  in  stan¬ 
dard  format  to  promote  repetitive  use  in  support  of  procurement,  pro¬ 
duction,  maintenance,  supply,  and  new  design. 

b  Specifications.  When  required  for  design  support,  configuration  identi¬ 
fication,  or  procurement,  an  adequate  engineering  definition  shall  be 
established  for  material  and  practices  resulting  from  new  development 
(engineering  and  operational  systems  development),  and  for  items 
authorized  for  production  or  supply  support.  The  preferred  format  is 
the  Federal  or  Military  specification. 
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c.  Industry  Documents.  Specifications  and  standards  of  nationally  recog¬ 
nized  industry  organizations  and  technical  societies  shall  be  used  in 
the  development  and  design  of  material,  and  in  the  preparation  of  Mili¬ 
tary  or  Federal  specifications  or  standards  to  the  maximum  extent 
practicable.  Duplication  in  the  Militaiy  series  of  industry  standards  is 
to  be  avoided. 

8.  Industry  Coordination.  The  procedures  for  Department  of  Defense  Stan¬ 
dardization  will  provide  for  industry  participation  to  an  appropriate  degree 
and  will  avoid  duplication  of  effort.  Consideration  will  be  given  to  the  con¬ 
tractual  responsibilities  of  contractors  in  the  design-development-produc¬ 
tion  cycle  to  prepare  required  documentation  and  also  to  the  needs  of 
contractors  in  the  use  of  documentation  produced  under  this  program. 


RESPONSIBILITIES  ASSIGNED  BY  THE  DSP 

The  Defense  Standardization  Program  assigns  responsibilities  as  follows. 

1.  Engineering  Determinations.  Design,  development,  and  engineering  activi¬ 
ties  are  responsible  for  the  engineering  determinations  recorded  in  specifi¬ 
cations,  standards,  drawings,  and  criteria  for  interchangeability  and 
substitutability  (when  the  criteria  are  not  contained  in  military  design 
standards). 

2.  Record  Preparation.  Design  development,  and  engineering  activities  are 
responsible  for  tho  timely  preparation  records  of  now  development  in 
authorized  format  for  the  support  of  configuration  management,  produc¬ 
tion,  procurement,  and  follow-on  logistics  support.  These  activities  also  are 
responsible  for  recording  and  approving  any  justified  use  of  nonstandard 
parts,  components,  and  materials. 

3.  Use  in  New  Design.  The  design  and  development  activities  have  responsi¬ 
bility  for  the  use  in  new  design  of  (a)  applicable  standards,  (b)  suitable 
items  available  in  supply,  and  (c)  suitable  commercial  items,  before  any 
new  development  is  authorized  to  meet  equipment  or  systems  objectives. 

4.  Users.  Users  of  engineering  information  are  responsible  for  the  formula¬ 
tion  of  programs  (e.g.,  item  reduction,  item  entry  control,  configuration 
management,  and  design  selection  discipline)  to  achieve  maximum  benefit 
from  use  of  standards  and  other  standardization  documents. 

5.  Logistics  Support.  Inventory  Managers  ore  responsible  for  programs  (a)  to 
screen  items  seeking  entry  into  supply  through  provisioning,  (b)  to  prevent 
identical  items  from  entering  with  differing  identifications,  and  (c)  to  sub¬ 
stitute,  for  the  new  item,  an  interchangeable  or  substitutable  item  already 
in  the  supply  system,  or  an  available  item  covered  by  a  military  standard 
or  specification.  Supply  management  is  responsible  for  limiting  the  num¬ 
ber  of  different  makes  and  models  of  equivalent  equipment  in  any  geo¬ 
graphical  area. 


XTv-9 


STANDARDIZATION  OF  COMPONENTS/ 
EQUIPMENTS  (C/E) 


One  of  the  Navy’s  standardization  efforts  is  outlined  in  NAVMATINST 
4120.97B.  In  this  instruction,  Components/Equipments  (C/E)  is  defined  as  repairable 
items  which  require  repair  part  support. 

The  Navy  Component/Equipment  Program  was  established  to  curb  the  prolif¬ 
eration  of  components/equipments  being  introduced  into  the  Fleet.  Several  logistics 
studies  had  reported  the  existence  of  many  nonstandard  and  low-population  equip¬ 
ments  requiring  differing  repair  parts  which  were  difficult  to  support.  Also,  such 
nonstandardization  has  contributed  directly  to  proliferation  of  allowance  parts  lists, 
technical  manuals,  configuration  management,  training,  and  other  logistic  require¬ 
ments. 

POLICIES  OF  THE  C/E  PROGRAM 

The  policy  of  the  C/E  Program  applies  to  all  elements  of  the  Navy  in  all 
phases  of  system  development,  acquisition,  and  maintenance.  It  includes  all  systems, 
subsystems,  components,  and  equipments.  Stated  in  the  broadest  terms,  the  policy  is 
to: 


1.  Include  hardware  standardization  requirements  in  concept  formulation, 
contract  definition,  procurement,  production,  maintenance,  conversion, 
modernization,  and  alteration. 

2.  Standardize  designs  —  with  intersystem  and  intrasystem  standardization 
of  C/E  and  parts. 

3.  Reuse  (in  new  design)  existing,  suitable  C/E  already  supported  in  depth  by 
the  military  system. 

4  Preclude  use  of  limited-application  and  poor-performance  C/E. 

5.  Exercise  configuration  control  to  maintain  standardization. 

6.  Use  procurement  techniques  to  restrain  proliferation. 

7.  Effect  item  entry  control  in  design  selection  and  provisioning  phases  of 
material  acquisition. 

C/E  PLAN  OF  OPERATION 

1.  The  plan  for  increasing  C/E  standardization  is  to: 

a.  Promulgate  and  implement  the  policy  set  forth  above. 

b.  Give  visibility  to  ongoing  standardization  effort  by  indexing  it. 
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c.  Promote  the  use  of  standardized  and/or  existing  C/E. 

d.  Backfit  standardization  into  existing  systems. 

2.  The  attainment  of  an  optimum  degree  of  standardization  by  curbing  C/E 
make  and  model  multiplication  and  resultant  spare  parts  proliferation 
must  be  within  the  bounds  of  the  Armed  Services  Procurement  Regulation 
(ASPR)  and  any  governing  requirements  of  the  Defense  Standardization 
Program.  Standardization  cannot  be  mandated  arbitrarily  but  must  result 
from  thoroughly  considered  tradeoffs,  generally  on  the  basis  of  cost  versus 
effectiveness. 

Each  Project  Manager  shall  maintain  his  own  internal  plan  for  standardi¬ 
zation  of  C/E  under  his  technical  cognizance  (NOTE:  Project  Managers 
will  not  be  required  to  develop  separate  standardization  plans  if  their 
PMPs  (Project  Master  Plans)  specifically  address  requirements  for  stan¬ 
dardization  of  C/E.)  Each  plan  shall  stipulate  the  development  of  indices  to 
reflect  the  current  situation  of  standardization  in  mqjor  commodity  areas 
(e.g.,  Hull/Mechanical/  Electrical,  Electronics  Communications  Equip¬ 
ment,  Electrical/Elcctronics  Test  Equipment,  Aviation  Ground  Support 
Equipment,  Avionics,  Aviation  Ordnance,  Conventional  Ordnance,  Guided 
Missiles)  against  which  to  plan  and  measure  future  accomplishment.  The 
plan  will  provide  for: 

a.  Implementing  the  standardization  poliqy  in: 

(1)  Concept  formulation,  contract  definition,  and  other  phases  of 
material  acquisition  planning,  including  Project  Master  Plans 
(PMPs),  Research  and  Development  Planning,  mid  Advanced  Pro¬ 
curement  Plans  (APPs). 

(2)  Procurement  of  C/E  with  use  of  standardization  requirements 
clauses,  where  warranted;  life-cycle  costing,  and  central  procure¬ 
ment  of  GFE/CFE  to  effect  consolidated  buys  of  standardized 
(identical  in  mako  and  model)  C/E.  The  Standardization  Exception 
(FAR  6.302-1),  covering  technical  equipment  requiring  standardi¬ 
zation  and  interchangeability  of  parts,  will  be  utilized  wherever 
the  stipulations  contained  therein  can  be  met  (for  tho  purposes  of 
application  of  tho  exception  a  “standard  item’  is  any  item  already 
in  use  in  tho  Navy  (e.g.,  an  item  identified  by  a  Federal  Stock 
Number  (FSN)  and/or  an  Allowance  Parts  List/Component  Identi¬ 
fication  (APL/CID)  number;  excluded  are  items  designated  non¬ 
standard  in  the  Federal  cataloging  records).  It  will  bo  specified 
that  all  systems/subsystems/components/equipments/instrumenta- 
tion  requiring  repair  part  support  are  to  bo  of  identical  mako  and 
model  (having  identical  internal  parts)  within  the  block  of  hard¬ 
ware  being  bought  under  any  individual  contract. 

b.  Developing,  whenever  practicable  standardized  design  for  C/E. 

c.  Selecting  C/E  for  new  systems  from  those  equipments  which  are  pres¬ 
ently  supported  (operationally  and  logistically)  in  depth. 
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d.  Utilizing  subsystems  of  one  system  in  other  systems  requiring  similar 
design  and  performance. 

e.  Assuring  that  a  minimum  variety  of  C/E  is  used  in  system  design  by 
incorporating  standard  C/E  with  better  performance,  or  oti:-»r  values, 
wherever  significant  logistics  benefits  will  accrue  and  can  be  measured 
on  a  life-cycle  cost  savings  basis. 


STANDARD  ELECTRONIC  MODULE  (SEM)  PROGRAM 


MIL-E-16400H,  via  MIL-STD-454  REQUIREMENT  73,  requires  the  consid¬ 
eration  of  the  SEM  packaging  techniques  conforming  to  MIL-STD-1378 

NAVMATINST  4120.102D  states  that  the  requirements  of  the  Standard  Elec¬ 
tronic  Module  Program  apply  to  ihe  initial  development  or  redesign  of  ship  and  shore 
electronic  equipment  and  systems.  The  program  is  optional  for  other  types  of  equip¬ 
ment  such  as  avionic  equipment,  satellites,  and  portable  equipment. 

All  PMs  will  participate  in  the  SEM  to  the  extent  that  SEM  modules  will  be 
used  in  new  systems  where  technically  and  economically  feasible.  The  features  of  the 
SEM  that  concern  maintaining  and  controlling  the  uso  of  the  standards  will  also  be 
obsorved.  The  instruction  requires  that  PMs  analyze  and  assess  the  feasibility  of 
using  SEM  modules;  and  that,  where  the  analysis  indicates  tho  SEM  modules  to  be 
technically  and  economically  suitable,  uso  of  the  modules  shall  be  specified  in  the 
development  of  the  system.  Once  this  decision  has  been  made,  the  SEM  procedural 
rules  established  by  SPAWAR  to  maintain  integrity  of  the  standards  shall  be 
observed.  The  requirements  do  not,  however,  relieve  responsibility,  nor  abrogate  the 
authority,  of  PMs  as  the  technical  and  management  agents  over  their  programs  in 
accordance  with  established  directives.  As  previously  stated,  the  project  manager  has 
the  option  of  using,  not  using,  or  discontinuing  use  of  SEM  modules  on  the  basis  of 
legitimate  and  demonstrable  teclmical  or  economic  factors.  Increased  participation  in 
the  SEM  is  desired,  and  will  be  of  mutual  benefit  to  all  concerned,  but  established 
authority  of  PMs  will  continue  to  be  recognized. 

PMs  who  require  testing  services  for  modules  under  a  specific  project  at  the 
SEM  Quality  Assurance  Activity  (NAD,  Crane)  will  be  responsible  for  the  cost  of  the 
testing  and  related  overhead. 

PMs  will  deal  directly  with  the  Quality  Assurance  Activity  through  normal 
channels.  Services  performed  by  the  Design  Review  Activity  (NAC,  Indianapolis)  that 
are  related  to  the  overall  SEM  standardization  program,  will  be  funded  by  SPAWAR 
and  made  available  to  other  SYSCOMs  and  PMs.  SPAWAR  will  provide  assistance  to 
other  SYSCOMs  and  PMs  in  applying  SEM. 

The  SEM  program  generally  achieves  Standardization  Advantages  1,  4-6, 
10-14,  16-18,  20,  21,  23,  and  28  while  suffering  Standardization  Disadvantages  3  and 

11. 
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RESPONSIBILITIES  UNDER  SEM 

All  PMs  will: 

1.  Review  all  planned  projects  within  the  scope  outlined  above  for  technical 
and  economic  applicability  of  the  SEM. 

2.  Where  review  indicates  feasibility  for  application  of  the  SEM,  require  its 
use  in  contract  specifications  by  citation  of  MIL-STD-1378. 

3.  For  those  programs  whose  combined  total  estimated  cost  for  R&D  and 
initial  production  exceeds  $10  million,  provide  notification  to  SPAWAR,  in 
format  of  RCS  NAVMAT  4120-10,  at  the  use  of  SEM  has  been  considered, 
if  not  used,  give  the  details  thereof. 

4.  Provide  for  the  production  sampling  tests  at  the  SEM  Quality  Assurance 
Activity  on  existing  standard  modules  at  will  be  used  in  the  project. 

5.  Provide  for  qualification  tests  at  the  SEM  Quality  Assurance  Activity  on 
new  standard  modules  that  will  be  needed  in  this  project 

6.  Provide  for  contractor  development  and  delivery  of  SEM  documentation 
for  new  standard  modules  resulting  from  the  project. 

7.  Recommend  to  SPAWAR  any  techniques  found  effective  in  specific  projects 
which  could  or  should  be  applied  Navy-wide. 

8.  Provide  points-of-contact,  technical  panel  representation,  and  supporting 
information,  upon  request,  to  assist  SPAWAR  in  SEM  management. 

STANDARDIZATION  REFERENCES 


1.  Naval  Electronic  Systems  Command 

MIL-M-2878C,  Modules  Electronic  Standard  Hardware  Program,  General 
Specification  for  1C  MAY  80 

2.  Naval  Electronic  Systems  Command 

MIL-STD-1378D,  Requirements  for  Employing  Standard  Electronic  Mod¬ 
ules,  16  May  86 

3.  Naval  Electronic  Systems  Command 

MIL-STD-1389C,  Design  Requirements  for  Standard  Electronic  Modules, 
16  May  86 

4.  Department  of  Defense  DoD  Directive  4120.3M 

5.  Secretary  of  the  Navy  SECNAVINST  4 120.3B, 

“Defense  Standardization  Program,”  25  Jan  68 
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0.  Naval  Material  Command  NAVMATINST  4120.97B, 
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7.  Naval  Material  Command  NAVMATINST  4120.1C2D,  “Standard  Elec¬ 
tronic  Modules  (SEM)  Program,”  20  Oct  82 
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XV.  WARRANTY  APPLICATIONS 


GENERAL 


The  use  of  warranties  in  contracts  is  intended  to  assure  the  buyer  of  —  and 
make  the  seller  liable  for  —  a  specified  performance  of  the  system/equipment  being 
procured.  For  government-purchased  electronic  equipment  such  assurance  has 
become  a  matter  of  great  concern  because  supporting  the  unreliable  performance  of 
many  of  these  items  has  become  increasingly  more  costly.  There  is  an  urgent  need  to 
reduce  support  costs  to  levels  that  are  consistent  with  today’s  stringent  funding  limi¬ 
tations,  and  DoD  is  taking  a  new  look  at  acquisition  strategy  as  one  way  to  solve  this 
cost  problem.  The  new  look  is  being  directed  in  part  at  the  use  of  types  of  warranties 
that  are  more  extensive  than  the  standard  warranties  for  which  FAR  has  clauses  and 
that  attack  the  support-cost  problem  directly. 

A  significant  portion  of  the  increasing  cost  of  maintenance  and  support  is  due 
to  poor  reliability.  It  is  axiomatic  that  the  greater  the  time  between  failures  (or  the 
fewer  failures),  the  less  the  support  cost  will  be  both  for  per-unit  cost  of  repair  and 
for  cost  of  spares.  Thus,  effective  effort  to  improve  reliability  of  an  equipment  will 
yield  reduced  support  costs,  not  to  mention  the  significant  benefit  of  improved  avail¬ 
ability. 


The  “new"  warranty  concept  the  government  is  considering  is  aimed  at  this 
reliability  mid  maintainability  improvement.  It  adopts  a  warranty  practice  the  airline 
industry  has  been  successfully  using  for  some  time  to  improve  reliability  and  lessen 
maintenance. 

Conventional  government  procurements  have  the  contractor’s  liability  ending 
with  delivery  and  government  acceptance,  with  reliability  and  maintainability  (R&M) 
accomplished  by  specifying  numerical  requirements  and  applicable  military-standard 
programs  during  design  development,  and  production.  The  efforts  have  been  partly 
successful,  but  have  v  jt  produced  the  desired  and  needed  results.  Too  often  the  gov¬ 
ernment  has  been  unable  to  exactly  define  reliability  requirements  and  measure  reli¬ 
ability  attainments.  This  has  made  it  difficult  to  levy  stiff  penalties  on  contractors  for 
failure  to  achieve  required  full  reliability,  and  has  resulted  ir.  deviations  and  waivers 
often  being  allowed  by  the  government  and  in  very  little  use  of  available  reliability- 
incentive  clauses  in  procurement  contracts. 

Working  to  aggravate  this  problem  is  the  contractor’s  inclination  to  maximize 
his  profits.  As  he  tries  to  improve  profits,  reliability  suffers  because  high  reliability 
costs  him  more  to  produce.  Regardless  of  the  intent  of  reliability  specification,  dem¬ 
onstration,  and  test  requirements,  the  contractor  has  the  final  say  about  the  reliabil¬ 
ity  level  of  the  item  being  produced  simply  because  he  is  the  one  actually  doing  the 
job.  And  if  he  is  motivated  more  by  profit  than  by  product  excellence,  the  delivered 
product  more  often  than  not  will  be  barely  acceptable  —  and  then  on  the  basis  of 
deviations  and  waivers. 

Enter  the  “Reliability  Improvement  Warranty’  (RIW).  The  RIW  is  one  of  the 
several  types  of  warranties  intended  for  improving  R&M  which  are  discussed  in  this 
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section,  and  one  to  which  DoD  is  giving  particular  attention  at  the  time  of  this  writ¬ 
ing.  Ths  RIW  works  to  take  advantage  of  contractor  profit  motivation  rather  than 
working:  at  cross  purposes  like  the  conventional  procurement  contract.  If  the  contrac¬ 
tor  is  made  responsible  for  long-term  reliability  after  dolivery  and  during  operation, 
and  for  a  fixed  fee  that  allows  him  to  increase  profits  in  proportion  to  the  level  of  reli¬ 
ability  and  the  ease  of  maintenance  that  are  achieved,  the  contractor  is  inclined  to 
bum  the  midnight  oil  on  his  R&M  programs.  The  RIW  becomes  a  contracting  tech¬ 
nique  by  which  the  government  derives  the  benefits  of  improved  reliability  and  main¬ 
tainability  for  each  individual  dollar  the  contractor  earns.  Other  warranties  of  this 
nature  approach  the  R&M  problem  a  little  differently,  as  will  be  noted. 

Before  discussing  the  individual  warranties  designed  to  lessen  the  cost  of 
R&M,  a  few  words  are  in  order  about  the  general  requirements  of  FAR  for  all  warran¬ 
ties  —  including  the  R&M  ones. 

Two  basic  types  of  warranties  exist  based  on  the  Uniform  Commercial  Code 
(UCC)  to  provide  buyer  protection.  No  federal  law  conflicts  with  the  UCC,  and  the 
government  has  adopted  it  for  government  contracting.  The  first  type  of  warranty  is 
an  implied  one.  In  government  procurements  the  existence  of  an  implied  warranty 
depends  on  the  type  of  contract;  it  does  not  apply  to  cost  reimbursement  contracts. 
The  second  type  of  warranty  is  an  expressed  one,  a  clause  being  included  in  the  con¬ 
tract  to  define  the  conditions  and  provisions  of  the  warranty.  Reliability  improve¬ 
ment  and  other  maintenance  and  repair-oriented  warranties  are  expressed 
warranties. 

Defense  Procurement  Circular  74-2,  issued  4  October  1974,  revised  the  War¬ 
ranty  Section  of  ASPR  and  added  new  clauses  for  conventional-type  warranties;  these 
were  carried  into  the  FAR.  The  new  clauses  do  not  differ  significantly  from  the  for¬ 
mer  except  for  a  major  change  which  states,  “All  implied  warranties  of  merchan¬ 
tability  and  fitness  for  a  particular  purpose  are  hereby  excluded  from  any  obligation 
contained  in  this  contract." 

Two  purposes  are  given  for  warranties:  to  delineate  rights  and  obligations 
regarding  defective  items,  and  to  foster  quality  performance.  FAR  policy  is  that  a 
warranty  clause  shall  be  used  whon  it  is  found  to  be  in  the  best  interest  of  the  gov¬ 
ernment. 

Now  provisions  involve  the  pricing  aspects  for  fixed-price  incentive  warranty 
provisions  and  differ  from  the  previous  FAR.  The  new  section  says  the  estimated 
costs  of  warranty  coverage  should  not  normally  be  considered  in  the  incentive  target 
price,  and  all  costs  should  be  considered  in  establishing  a  ceiling  price.  All  warranty 
costs  incurred  are  to  be  considered  then  in  the  total  final  price.  And  after  establish¬ 
ment  of  the  total  final  price,  the  contractor  is  to  bear  all  warranty  costs. 


RELIABILITY  IMPROVEMENT  WARRANTY  (RIW) 


DoD  has  requested  the  Armed  Services  to  initiate  RIWs  on  a  trial  basis,  to 
help  determine  the  scope  and  benefits  that  these  warranties  may  have.  In  a  memoran¬ 
dum  signed  by  both  the  Assistant  Secretary  of  Defense  (I&L)  and  the  Director  of 


XV-2 


1 

Defense  Research  and  Engineering,  guidelines  for  a  test  program  have  been  provided 
for  trial-basis  use.  These  guidelines  are  similar  to  those  published  by  the  Air  Force  in 
July  1974  in  Interim  Guidelines  Reliability  Improvement  Warranty  (RIW). 

An  equipment  acquisition  contract  having  an  RIW  provision  specifies  equip¬ 
ment  characteristics  for  form,  fit,  and  function  only,  and  allows  the  contractor 
freedom  of  design  as  well  as  freedom  to  change  design  without  intervention  of  all  the 
usual  government  configuration-management  constraints,  but  within  government 
requirements  for  preserving  configuration  control.  The  RIW  provision  stipulates  that 
the  contractor  will,  for  a  certain  length  of  time  or  number  of  operating  hours,  repair 
and/or  replace  the  units  in  service  upon  occasion  of  certain  defined  failures. 

A  firm  fixed-price  contract  is  used,  one  price  including  both  acquisition  and 
follow-on  servicing.  Once  a  fixed  price  is  established  for  the  contract,  the  actual  profit 
realized  by  the  contractor  depends  on  the  equipment’s  reliability  and  maintainability 
in  service  rise,  plus  any  improvements  that  he  can  make  in  R&M  during  the  warranty 
period  to  keep  the  number  and  cost  of  repairs  as  low  as  possible.  In  this  connection, 
the  terms  and  commitments  required  of  the  contractor  by  the  RIW  provision  must 
result  in  a  reasonable  balance  between  his  risks  and  the  degree  of  incentive  (profits) 
needed  to  achieve  the  primary  goal  of  system/equipment  availability.  Such  considera¬ 
tion  must  include  the  uncertainties  of  future  support  costs  and  the  resultant  risks  to 
both  contractor  and  government. 

The  RIW  is  not  the  same  as  a  maintenance  contract  and  does  not  require  the 
contractor  to  provide  routine  periodic  upkeep,  regulation,  adjustment,  cloaning,  or 
other  normal  upkeep.  Government  personnel  accomplish  these  jobs.  The  RIW  also 
does  not  cover  components  of  the  warranted  item  which  usually  need  replacement 
under  normal  use;  such  items  may  be  covered  by  separate  provisions  in  the  contract, 
but  cannot  be  included  in  the  RIW  provision. 

ADVANTAGES 


When  a  new-procurement  item  meets  criteria  for  IUW  application  (to  bo  dis¬ 
cussed),  the  use  of  the  RIW  offers  the  following  advantages  to  the  government: 

1.  Contractor  has  responsibility  and  incentive  for  improving  field  reliability. 

2.  Financial  commitment  for  logistic  support  is  known  and  constant  during  a 
3-5  year  period  of  possible  economic  inflation. 

3.  Life-cycle  cost  approach  can  receive  greater  emphasis. 

4.  Contractor  has  responsibility  for  configuration  management  of  field  units 
and  for  keeping  all  units  to  same  configuration. 

5.  Contractor  has  an  incentive  to  introduce  design  and  production  changes 
that  will  increase  MTBF  and  result  in  reliability  growth. 

6.  Contractor  has  an  incentive  to  introduce  design  and  production  changes 
that  will  reduce  labor  hours  and  materials  needed  for  field  repairs. 

7.  Minimal  initial  support  investment  is  required. 
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8.  Requirements  for  skilled  military  maintenance  and  support  manpower  are 
reduced. 

Use  of  the  RIW  offers  the  following  advantages  to  the  contractor: 

1.  Increased  profit  potential  when  field  MTBF  is  increased  above  contractual 
MTBF. 

2.  Guaranteed  multiyear  business. 

3.  More  familiarity  with  operational  reliability  and  maintainability  charac¬ 
teristics,  which  is  advantageous  in  obtaining  additional  government  con¬ 
tracts. 

Benefits  obtained  from  the  RIW  concept  can  be  maximized  only  if  prospective 
contractors  are  notified  early  in  the  development  period  that  the  government  intends 
to  consider  including  such  warranty  provision  in  the  production  contract.  Otherwise, 
the  contractor  will  not  have  the  incentive  intended  by  the  RIW  to  give  necessary 
attention  to  R&M  at  the  time  of  initial  design,  the  outcome  of  which  has  significant 
effect  on  eventual  MTBF  and  repair  costs. 

USING  THE  RIW 

The  use  of  RIWs  is  limited  to  certain  acquisitions  of  equipment  that  satisfy  (1) 
stipulations  by  ASPR  for  warranty  use  in  gonoral,  (2)  spocific  criteria  for  RIW  use,  (3) 
certain  funding  requirements,  and  (4)  conditions  relating  to  elements  that  must  be 
contained  in  the  RIW  clause  of  a  contract.  These  sets  of  conditions  are  presented  in 
the  following  paragraphs. 

FAR/DAR  Requirements  for  General  Warranty  Use 

FAR  52.246  lists  the  following  factors  that  must  be  considered  in  deciding 
whether  a  warranty  clause  of  any  kind  is  to  be  used  in  a  contract. 

1.  Nature  of  the  item  and  its  end  use 

2.  Cost  of  the  warranty  and  degree  of  price  competition  as  it  may  affect  this 
cost 

3.  Criticality  of  meeting  specifications 

4.  Damages  to  the  government  that  might  be  expected  to  arise  in  the  event  of 
defective  performance 

5.  Cost  of  correction  or  replacement,  either  by  the  contractor  or  another 
source,  in  the  absence  of  a  warranty 

6.  Administration  cost  and  difficulty  of  enforcing  the  warranty 

7.  Ability  to  take  advantage  of  the  warranty,  as  conditioned  by  storage,  time, 
distance  of  the  using  agency  from  the  source,  or  other  factors 
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8.  Operation  of  the  warranty  as  a  deterrent  against  deficiencies 

9.  The  extent  to  which  government  acceptance  is  to  be  based  upon  contractor 
inspection  or  quality  control 

10.  Whether  because  of  the  nature  of  the  item  the  government  inspection  sys¬ 
tem  would  not  be  likely  to  provide  adequate  protection  without  a  warranty 

11.  Whether  the  contractor’s  present  quality  program  is  reliable  enough  to 
provide  adequate  protection  without  a  warranty 

12.  Reliance  on  “brand  name’  integrity 

13.  Whother  a  warranty  is  regularly  given  for  a  commercial  component  of  a 
more  complex  end  item 

14.  Criticality  of  item  for  protection  of  personnel  or  property;  e.g.,  for  safety  in 
flight 

15.  The  stage  of  development  of  the  item  and  the  state  of  the  art 

16.  Customary  trade  practices 

FAR  52.246-16, 17, 18,  and  19  cover  the  warranty  clauses  most  generally  of  interest 
in  system  acquisitions. 

Specific  Criteria  for  RIW  Use 

Currently  there  are  three  sources  for  criteria  which  establish  tho  practicabil¬ 
ity  or  advisability  of  using  the  RIW.  They  are  the  joint  memorandum  of  ASD(I&L)/ 
DDR&E;  tho  Air  Force  interim  guidelines  document  for  RIWs;  and  a  tabulation  of 
criteria  developed  by  ARINC  Research  Corporation  for  the  Rome  Air  Development 
Center  All  throo  cover  essentially  the  same  points.  The  ARINC  tabulation,  however, 
is  more  precise  and  definitive  and  is  presented  here  as  table  XV-1. 

Throe  broad  areas  of  consideration  are  to  be  found  in  the  tabulated  criteria: 
procurement  factors,  equipment  characteristics,  and  application  factors.  Each  is 
equally  important  in  accepting  or  rejecting  RIW  use.  The  individual  factors  within 
each  area,  however,  do  vary  in  importance,  and  have  been  ranked  as  follows: 

“I’  factors  are  of  major  importance.  Failure  to  meet  tho  stated  criterion  could 
be  grounds  for  not  using  the  RIW. 

*2’  factors  are  of  secondary  importance.  Failure  to  meet  the  criterion  of  one  of 
these  factors  will  generally  not  be  sufficient  in  itself  for  rejecting  the  RIW, 
but  a  combination  of  “2"  factors  could  be. 

“3"  factors  are  of  minor  importance.  Failure  to  meet  these  criteria  is  generally 
not  considered  serious,  but  may  require  special  consideration  in  structur¬ 
ing  the  warranty  or  the  administrative  procedures. 
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Table  XV-1.  Criteria  for  use  of  RIWs  in  hardware  acquisition  contracts. 


Major,  2  —  Secondary;  3  —  Minor 


1  -  Major;  2  -  Secondary;  3  -  Minor 


Major;  2  -  Secondary;  3  —  Minor 
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1  -  Mjgor;  2  -  Secondary;  3  -  Minor 


It  should  be  emphasized  that  the  criteria  in  table  XV-I  are  to  be  used  qualita¬ 
tively  with  an  awareness  of  the  extra  effort  and  cost  required  of  both  the  government 
in  requesting  and  the  contractor  in  proposing  the  RIW  provision.  That  is,  there 
should  be,  before  requesting  contractors’  proposals,  a  reasonable  certainty,  based  on 
cost  analysis  as  well  as  the  criteria  factors,  that  the  RIW  will  be  employed,  even 
though  a  thorough  economic  analysis  of  the  use  cannot  be  made  until  RIW  price  and 
implementation  proposals  are  received  from  the  bidding  contractors.  To  decide  on  the 
basis  of  cost  whether  or  not  the  RTW  may  be  a  good  course  of  action,  there  must  be  a 
MTBF  figure  on  which  to  base  a  probable  price  for  contractor  development/produc- 
tion  and  support  (acquisition  with  RIW),  and  then  this  price  must  be  compared  with 
the  price  of  a  development/production-only  contract  plus  the  probable  cost  of  support 
by  the  government  without  the  RIW.  This  may  be  undertaken,  as  indicated  in  figure 
XV-I,  by  plotting  costs  versus  a  range  of  MTBF  levels  for  development/production 
(curve  A)  and  operation/maintenance  (curve  B),  which  curves  when  added  provide 
rough  figures  for  a  life-cycle  cost  (curve  C).  The  vertical  width  of  the  curves  in  figure 
XV-1  represents  the  uncertainty  of  the  costs  associated  with  achieving  and  supporting 
the  various  levels  of  reliability.  The  use  of  life-cycle  cost  (LCC)  models  can  assist  in 
this  evaluation  process  when  sufficient  data  are  available. 

When  judgment  has  been  made  for  the  use  of  the  RIW  provision  in  the  RFP/ 
contract,  the  RFP  is  worded  to  indicate  that  the  RIW  provision  may  or  may  not  be 
included  in  the  contract,  depending  on  responding  proposals  and  what  they  have  to 
say  about  proposed  MTBF  and  cost. 

Upon  receipt  of  bidders'  proposals,  further  cost  analysis  is  made  to  determine 
whether  the  use  of  the  RTW  would  be  cost-effective.  The  life-cycle  cost  curve  devel¬ 
oped  prior  to  RFP  release  —  such  as  the  one  shown  in  figure  XV-1C  —  can  be  com¬ 
pared  with  the  total  fixed-price  bids  in  the  proposals  and  their  respective  proposed 
MTBFs.  If  a  cost  proposal  is  below  the  lower  bound  of  the  cost  curve,  the  price  is 
right  and  use  of  the  RIW  is  definitely  indicated.  If  the  proposal  is  above  the  upper 
bound,  the  RIW  should  not  be  considered.  In  between,  further  evaluation  must  be 
made.  The  fair-price  question  here  is  helped  considerably,  os  is  achievable  reliability, 
by  tho  fact  of  competition  among  bidders  for  a  fixed-price  contract.  Overestimation  of 
expected  reliability  by  a  bidder  will  tend  to  increase  his  total  price  and  reduce  his 
chance  of  being  accepted,  whereas  underestimation  for  the  purpose  of  improving 
profit  may  lose  him  tho  contract  as  well. 

A  bidder  may  be  inclined  to  lump  the  costs  associated  with  the  RIW  into  his 
unit  price.  But,  in  order  to  make  an  accept/reject  decision  on  the  use  of  the  RTW,  the 
actual  price  proposed  for  the  RIW  must  be  known.  Bidders  therefore  must  be  required 
to  separately  prico  the  RIW  provision  so  that  a  clear  comparison  may  be  made  with 
the  government's  cost  estimate.  The  accept/reject  decision  is  based  in  no  small  part 
on  the  difference  between  the  RIW  support  costs  and  the  support  costs  the  govern¬ 
ment  would  incur  if  the  equipment  were  purchased  without  an  RIW.  The  warranty 
should  normally  cost  4-6%  of  the  operational  unit  acquisition  cost  per  year. 
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MTBF 

A.  COST  OF  RELIABILITY  IMPROVEMENT  (DEVELOPMENf/PRODUCTION) 


MTBF 

B.  OPERATION  AND  MAINTENANCE  COST 


MTBF 

C.  UFE-CYCLE  SUPPORT  COST 

Figure  XV-1.  Development  of  life-cycle  support  cost. 
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Funding  Requirements 

To  clarify  the  types  of  funds  to  be  used  for  procurements  having  the  RTW  pro¬ 
vision,  the  following  funding  policy  guidelines  have  been  authorized  for  use  by 
OASD(C)  and  OAGCfFM): 

1.  RTWs  shall  be  funded  from  the  same  appropriation  as  the  acquisition.  That 
is,  the  RTW  shall  be  paid  from  the  procurement  or  RDT&E  appropriation 
of  the  service  or  agency  concerned,  depending  on  from  which  of  the  appro¬ 
priations  the  acquisition  is  funded. 

2.  The  RIW  price  shall  be  part  of  the  fixed-price  contract,  and  payment  to  the 
warrantor  for  RIW  portion  shall  not  be  made  differently  from  payment 

under  the  remaining  portion  of  the  contract  except  that  payment  for  the 
RIW  may  be  delayed  until  delivery  or  relinquishment  of  production  control 
of  the  item  by  the  warrantor. 

In  addition,  to  maintain  the  distinction  between  an  RIW  and  a  service  con¬ 
tract  covering  normal,  periodic  maintenance,  the  following  requirements  must  be  sat¬ 
isfied: 

3.  The  warranty  period  on  the  item(s)  shall  begin  after  manufacture  and  upon 
deliveiy  or  relinquishment  of  production  control  by  the  warrantor. 

4.  The  RIW  shall  require  the  warrantor  to  repair  or  replace  the  warranted 
item  upon  failure. 

5.  The  RIW  shall  not  include  requirements  for  the  warrantor  to  provide  nor¬ 
mal  upkeep,  cleaning,  ac(justing,  regulating,  or  other  periodic  maintenance 
accomplished  whethor  or  not  failure  occurs. 

6.  The  RIW  shall  exclude  components  of  the  warranted  item(s)  which  nor¬ 
mally  require  replacement  during  the  period  of  the  warranty  (such  as  fil¬ 
ters  and  light  bulbs).  These  components  may  bo  accounted  for  by  separate 
provisions  in  the  contract  consistent  with  current  laws  and  regulations. 

Essential  Elements  in  the  RIW  Clause 

Because  RIW  provisions  must  be  tailored  to  the  particular  item  being  war¬ 
ranted,  a  standard  RIW  clause  is  not  contained  in  the  FAR.  However,  certain  ele¬ 
ments  should  be  considered  for  inclusion  in  an  RTW  clause.  These  elements,  which 
are  discussed  below,  concern  (1)  the  statement  of  contractor  warranty,  (2)  contractor 
obligations,  (3)  government  obligations,  (4)  miscellaneous  requirements,  and  (5)  data 
requirements.  RIW  provisions  are  constructed  by  implementing  tailored  provisions 
under  “Warranty  of  Supplies  of  a  Complex  Nature”  (FAR  52.246-18). 

Statement  of  Contractor  Warranty 

1.  Term.  State  the  length  of  time  the  warranty  will  be  in  effect.  Usually  this 
should  cover  usage  (operating  hours)  and  calendar  time  (generally  3  to  5 
years),  whichever  occurs  first. 
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2.  Ohiective/scone.  State  the  primal  objective  of  the  warranty;  that  is,  to 
motivate  the  contractor  to  design  and  produce  equipment  that  is  more  reli¬ 
able  and  less  costly  to  repair.  If  there  is  a  specified  reliability  requirement, 
it  should  be  clearly  set  forth. 

3.  Failure.  Define  the  failure(s)  which  ill  require  the  contractor  to  repair  or 
replace  a  failed  item  at  no  change  in  the  contract  price,  when  the  item  is 
returned  to  him. 

4.  Exclusions.  State  the  conditions  and  actions  associated  with  repair/ 
replacement  which  are  specifically  excluded  under  the  warranty  —  such  as 
items  lost  or  damaged  due  to  fire,  explosion,  packing,  and  shipping. 

5.  Shinping  costa  State  who  pays  the  costs  of  shipping  the  failed  units  to  the 
contractor,  the  government  or  the  contractor. 

6.  Price.  State  that  a  price  breakout  of  the  warranty  should  be  included 
(along  with  the  total  price)  to  allow  the  government  to  determine  the  cost 
of  the  RIW. 

Contractor  Obligations 

1.  Warranty  markings.  State  that  the  contractor  shall  be  required  to  promi¬ 
nently  display  the  following  information  on  the  face  of  the  unit:  that  the 
item  is  warranted;  the  warranty  period;  and  actions  to  take  if  the  unit  fails 
du-ing  the  warranty  period. 

2.  Turnaround  time.  State  the  turnaround  time,  and  the  appropriate  adjust¬ 
ments  or  other  considerations  to  be  exacted  if  the  contractor  exceeds  the 
number  of  days  so  specified.  A  contract  turnaround  time  should  be  defined 
as  the  period  between  the  date  the  unit  is  received  by  the  contractor  for 
repair/replacement  and  the  date  when  the  repaired/replacing  unit  is 
shipped  by  the  contractor  to  the  government. 

3.  Records.  State  that  the  contractor  shall  maintain  records,  by  serial  num¬ 
ber,  for  each  unit  under  warranty,  and  shall  make  such  records  available  to 
tho  government  upon  request. 

Government  Obligations 

1.  Containers.  State  whether  or  not  the  government  will  supply  special  con¬ 
tainers  for  transporting  units  to  and  from  their  destinations  for  the  life  of 
the  warranty. 

2.  No-cost  modifications.  State  the  procedures  for  submittal  of  contractor 
initiated  no-cost  Engineering  Change  Proposals  (ECPs)  (designed  to 
improve  the  item’s  reliability/maintainability).  The  contractor  should  be 
advised  that  such  ECPs  will  be  subject  to  government  approval. 

Miscellaneous 

1.  Inspection.  State  the  extent  of  both  government  and  contractor  inspection 
to  be  required. 
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2.  Dispositioa  State  that  each  unit  returned  to  the  contractor  which  is  con¬ 
sidered  to  be  nonrepairable  shall  be  imposed  of  by  the  contractor  as 
directed  by  the  government.  Also,  state  the  banner  of  disposition  of  the 
unused  portion  of  the  warranty  for  such  nonrepairable  unit  as  well  as  for 
other  similar  instances  such  as  when  the  government  has  certified  that  a 
unit  has  been  lost. 

3.  Notification.  State  the  requirements  including  time  limits,  for  both  the 
government  and  contractor  to  notify  each  other  of  deficiencies  in  a  unit. 

4.  Unverified  failures.  State  whether  or  not  the  contractor  will  be  compen¬ 
sated  for  the  cost  of  testing  units  returned  to  him  under  the  warranty 
when  the  tests  reveal  no  discrepancies. 

5.  Adjustments.  State  the  circumstances,  if  any,  under  which  the  government 
is  authorized  to  make  adjustments  to  units  under  the  warranty. 

Data  Requirements 

1.  Contractor  warranty  data.  State  that  the  contractor  shall  establish  and 
maintain  a  data  system  that  will  provide  formation  on  the  repair  record  of 
each  unit,  analysis  of  unit  failure,  number  of  units  returned,  turnaround 
and  pipeline  times  of  units  returned,  remaining  warranty  coverage,  etc. 

2.  Government-developed  data  State  that  the  government  will  be  required  to 
provide,  in  a  timely  manner,  available  government-generated  operation 
and  maintenance  data  pertaining  to  the  warranted  units. 

Summary  of  RIW  Use 

Even  to  considor  the  use  of  the  RIW  in  a  procurement  contract  depends  on 
whether  the  item  is  expected  to  have  moderate  to  high  support  costs.  If  so,  then  the 
item  must  be  amenable  to  being  specified  only  in  terms  of  form,  fit,  and  function, 
without  the  need  for  detail  design  constraints.  When  these  conditions  prevail,  then 
the  four  measures  of  suitability  discussed  above  can  be  applied.  And  the  item  meas¬ 
ures  up,  the  project  manager  has  the  green  light  to  go  ahead  with  the  RIW. 


OTHER  R&M  COST-REDUCTION  WARRANTIES 


Three  other  types  of  R&M  cost-reduction  warranties  are  presented.  They  are 
referred  to  as  Mean  Time  Between  Failure,  Equipment  Turnaround  Time,  and  Main¬ 
tenance  Cost  —  names  that  indicate  the  nature  of  the  guarantee  being  purchased. 
They  are  warranties  with  which  the  airline  industry  has  had  some  success,  but  with 
which  the  government  has  had  little  experience.  Consequently,  their  use  as  allowed 
by  FAR  must  be  carefully  considered,  just  as  the  use  of  the  Reliability  Improvement 
Warranty  must  be  carefully  considered,  and  their  clauses  must  be  tailored  to  the  par¬ 
ticular  procurement  requirements.  Clauses  that  have  been  used  by  the  airline  indus¬ 
try  are  presented  for  each  of  these  warranties  as  examples  only  of  the  elements  to  be 
considered  for  inclusion  in  government  clauses. 
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MEAN  TIME  BETWEEN  FAILURES  (MTBF)  WARRANTIES 


Under  this  type  of  warranty  the  contractor  guarantees  that  the  equipment  will 
achieve  a  stated  MTBF.  If  the  equipment  fails  to  do  so,  the  contractor  must  provide 
labor  and  materials  to  modify  the  equipment  to  meet  the  MTBF  requirements.  In 
essence,  this  is  another  way  to  obtain  a  required  level  of  reliability. 

An  MTBF  warranty  can  reduce  infant  mortality  because  it  gives  the  contrac¬ 
tor  an  incentive  to  test  the  equipment  before  delivering  it  to  the  government.  On  the 
other  hand,  historical  information  on  the  equipment  must  be  made  available  to  deter¬ 
mine  an  equitable  and  realistic  price.  The  contract  also  must  contain  definitions  for 
various  classes  of  failures  which  must  be  mutually  satisfactory  to  both  parties.  Also, 
conditional  MTBF  warranties  can  lead  to  disputes;  an  unconditional  warranty  under 
which  all  agreed-upon  failure  conditions  are  rectified  is  more  desirable,  but  also  more 
expensive. 

Examples  of  Clauses 

1.  Each  manufacturer  guarantees  that  the  Equipment  covered  under  this 
agreement  shall  achieve  the  Mean  Time  Between  Failure  (MTBF)  guaran¬ 
tees  as  set  forth  in  Attachment  XXX,  a  copy  of  which  is  attached  hereto 
and  made  a  part  hereof. 

2.  Buyer  shall  provide  Equipment  failure  data  from  the  date  the  Equipment 
enters  the  Buyer’s  service.  The  data  shall  be  sufficient  to  determine  MTBF 
and  any  additional  equipment  required. 

3.  Equipment  provisioning  shall  be  determined  by  Buyer  and  shall  be  based 
upon  the  MTBF  guarantee  set  forth  in  Attachment  XXX  as  modified  by 
other  program  factors  determined  by  Buyer.  MTBF  guarantees,  set  forth  in 
Attachment  XXX,  for  the  purpose  of  this  program,  shall  be  divided  into 
three  periods:  (a)  Initial  —  first  24  months;  (b)  Interim  —  25th  to  42nd 
months;  and  (c)  Final  —  43rd  and  subsequent  months. 

4.  Support  is  to  be  based  upon  the  data  provided  by  Buyer  in  accordance  with 
paragraph  2  above,  and  such  data  shall  be  used  to  compute  any  additional 
equipment  required.  Such  equipment  hall  be  made  available  on  a  no-charge 
consignment  basis. 

5.  MTBF  measurements  shall  be  based  upon  a  monthly  measurement  corre¬ 
sponding  to  a  3-month  moving  average.  The  Manufacturer’s  obligation 
under  the  MTBF  Guarantee  Program  shall  commence  on  the  date  the 
Equipment  enters  Buyer's  service  and  shall  terminate  when  the  respective 
MTBF  guarantees  set  forth  in  Attachment  XXX  are  achieved  over  18  con¬ 
secutive  monthly  measurements  commencing  no  earlier  than  the  43rd 
month  after  introduction  of  the  Equipment  into  Buyer’s  service. 
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6.  The  specific  provision  for  measuring  the  MTBF  is  as  hereinafter  set  forth: 

a.  Calculation  of  MTBF:  The  MTBF  shall  be  calculated  by  the  application 
of  the  following  formula: 

Cumulative  Number  The  Number  of 

of  Hours  of  Equip-  x  Equipments 

ment  Operation 

MTBF  =  Total  Cumulative  Number  of  Chargeable  Failures 

b.  Definition  of  Failure:  The  following  failure  definitions  and  conditions 
shall  apply: 

(1)  Confirmed  Failure:  Any  Equipment  removed  from  a  platform  for 
suspected  failure  shall  be  deemed  a  confirmed  failure  when,  upon 
being  subjected  to  test  in  the  condition  removed,  it  is  unable  to 
pass  the  test  for  the  Equipment  specified  by  Manufacturer’s  Over¬ 
haul  Manual  supplied  to  Buyer  or  other  mutually  agreeable  test 
procedure.  The  specified  test  must  be  comparable  in  scope  to 
Manufacturer’s  acceptance  test  for  production  equipment.  Tests 
may  be  performed  in  Buyer’s  facilities  or  those  of  its  approved 
designee. 

(2)  Irrelevant  Failures:  Irrelevant  failures  shall  not  be  counted  in  the 
MTBF  determination.  Irrelevant  failures  are  defined  as  follows: 

(a)  A  failure  caused  by  a  condition  external  to  the  Equipment, 
such  as  improperly  supplied  power,  improper  interconnecting 
wiring,  or  improper  operation  of  tho  Equipment. 

(b)  The  failure  is  a  dependent  (secondary)  failure  resulting  from 
an  independent  (primary)  failure  within  the  same  Equipment, 
provided  that  the  independent  (primary)  failure  is  specified  A 
dependent  failure  occurring  in  a  separate  piece  of  equipment 
from  tho  Equipment  which  the  primary  confirmed  failure 
occurred  shall  be  considered  as  a  confirmed  failure. 

(3)  Additional  Requirements:  At  all  times  while  in  Buyer’s  possession, 
the  Equipment  shall  be  subjected  to  an  environment  within  Speci¬ 
fication  requirements.  Failures  which  occur  as  a  result  of  an  expo¬ 
sure  to  an  environment  in  excess  of  that  specified  shall  not  count 
in  the  MTBF  determination.  Failures  resulting  from  accident  or 
improper  maintenance  shall  not  count  in  the  MTBF  determina¬ 
tion.  Operation  and  maintenance  procedures  shall  be  in  accor¬ 
dance  with  the  applicable  operating  and  maintenance  manuals. 

7.  a.  In  the  event  tho  MTBF  calculated  for  any  Equipment  in  operation 
in  a  calculation  period  is  less  than  the  guaranteed  MTBF,  Manu¬ 
facturer  shall  re-engineer  and  modify  such  Equipment  as  required 
to  meet  the  guaranteed  MTBF,  at  no  charge  to  Buyer,  and  consign 
additional  Equipment  at  no  charge  based  on  the  following  formula: 
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where 


n  =  KN 


( M-m ) 
Mm 


n  =  Maximum  number  of  additional  Equipment  to  be 

consigned  to  Buyer  under  MTBF  Guarantee  Pro¬ 
gram.  This  number  shall  be  rounded  to  the  near¬ 
est  whole  number,  but  not  less  than  1,  and  shall 
not  exceed  100%  of  Equipment  installed  in  Buy¬ 
er’s  platforms  as  of  the  date  of  MTBF  calculation. 


K  = 

5000 

N  = 

Total  number  of  Equipment  installed  in  Buyer’s 
platforms  as  of  the  date  of  MTBF  calculation. 

M  = 

Guaranteed  MTBF  for  the  Equipment. 

m  — 

Calculated  average  MTBF  of  the  Equipment. 

b.  Failuro  classification  shall  be  mutually  agreeable  to  Manufacturer  and 
Buyer.  If  no  agreement  can  be  reached,  the  failed  Equipment  shall  be 
subject  to  failure  analysis  prior  classification. 

c.  If  additional  consignment  Equipment  is  required  to  be  furnished  by 
Manufacturer  to  Buyer  hereunder,  Manufacturer  shall  ship  such 
Equipment  to  Buyer  not  later  than  1  week  after  completion  of  the 
MTBF  calculation  by  Buyer.  Buyer  shall  notify  Manufacturer  if  the 
indicated  number  of  consignment  Equipment  exceeds  Buyer’s  require¬ 
ments,  in  which  caso,  Manufacturer  shall  bo  obligated  to  supply  only 
that  quantity  required  by  Buyer.  Manufacturer  agrees  to  furnish  each 
additional  Equipment  notwithstanding  the  possible  existence  of  any 
disagreement  as  to  failure  classification. 

8.  Roturn  of  Consigned  Equipment:  Any  Equipment  consigned  under  the  pro¬ 
visions  of  paragraph  7  above  shall  be  returned  to  Manufacturer  by  Buyer 
not  later  than  90  days  after  an  MTBF  calculation.  Buyor  shall  have  the 
flexibility  of  replacing  any  consigned  Equipment  with  similar  Equipment. 

EQUIPMENT  TURNAROUND  TIME  WARRANTIES 


Under  the  terms  of  this  warranty  the  manufacturer  guarantees  to  repair  his 
delivered  equipment  within  a  stipulated  average  turnaround  time.  Normally,  it  is  as¬ 
sumed  tliat  the  same  unit  will  be  returned  to  the  government  after  repair,  but  allow¬ 
ances  can  be  made  to  replace  the  unit  with  another  like  unit  to  expedite  turnaround 
time. 

This  type  of  warranty  is  especially  suited  to  depot-level  or  manufacturer 
repair  situations.  Only  a  limited  area  of  concern  is  covered,  however.  Other 
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guarantees  would  have  to  be  used  to  cover  reliability  as  well  as  the  costs  of  mainte¬ 
nance  and  repair.  This  warranty  would  have  to  be  tailored  if  it  was  to  be  used  for 
shipboard  equipment,  which  should  not  be  too  difficult. 

Examples  of  Clauses 

1.  Manufacturer  guarantees  the  average  repair  turnaround  time  as  listed  in 
Attachment  XXX,  a  copy  of  which  is  attached  hereto  and  made  a  part 
hereof,  for  the  Equipment  covered  under  this  Agreement  requiring  repair 
or  correction.  These  turnaround  times  include  round-trip  shipping  time 
between  Buyer’s  maintenance  facilities  and  Manufacturer’s  repair  facility. 
The  average  turnaround  time  shall  be  calculated  on  a  3-month  moving 
average.  In  the  event  Manufacturer  fails  to  meet  the  average  turnaround 
times  as  listed  in  Attachment  XXX,  Manufacturer  shall  consign  additional 
Equipment,  at  no  charge,  in  accordance  with  the  following  formula: 

N  =  R(t-  T) 

where 

N  =  Number  of  additional  Equipment  to  be  consigned  to  Turn¬ 
around  Time  Guarantee.  This  number  shall  be  rounded  to  the 
nearest  whole  number,  but  shall  not  be  less  than  1. 

R  =  Quantity  of  Equipment  returned  to  Manufacturer  for  repair,  in 
units  per  week,  calculated  on  a  3-month  moving  average. 

T  =  Guaranteed  repair  turnaround  time  in  weeks 

t  =  Calculated  average  repair  turnaround  time  in  weeks. 

2.  The  above  assumes  that  the  same  serial  numbered  Equipment  shall  be 
returned  to  Buyer  as  received.  However,  Manufacturer  shall  have  flexibil¬ 
ity  to  replace  the  failed  Equipment  with  other  similar  equipment,  if  this 
will  expedite  repair,  provided  that  the  number  of  hours  on  such  other 
equipment  shall  not  exceed  the  number  on  such  failed  equipment 


MAINTENANCE  COST  WARRANTIES 


Under  this  warranty  the  contractor  guarantees  that  the  labor  and  material 
costs  for  the  government  to  maintain  the  equipment  will  not  exceed  a  specified 
amount.  If  this  amount  is  exceeded,  the  contractor  will  reimburse  the  government  for 
the  excess. 

Use  of  this  type  of  warranty  allows  the  government  to  use  its  own  mainte¬ 
nance  personnel.  The  warranty  could  be  practicable  for  use  with  shipboard  equip¬ 
ment.  However,  a  maintenance  history  of  the  equipment  is  required  to  determine  an 
equitable  and  realistic  contract  price  and,  during  application,  thorough  maintenance- 
cost  records  must  be  kept. 
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Examples  of  Clauses 


1.  Manufacturer  guarantees  that  the  maintenance  labor  and  material  cost  for 
Buyer  to  maintain  the  Equipment  covered  under  this  Agreement  shall  not 
exceed  the  respective  guaranteed  maintenance  labor  man-hour  costs  and 
maintenance  material  costs  set  forth  on  Attachment  XXX,  a  copy  of  which 
is  attached  hereto  and  made  a  part  hereof.  The  Maintenance  Cost  Guaran¬ 
tees  set  forth  in  Attachment  XXX  all  commence  with  delivery  of  the  first 
unit  to  Buyer  and  shall  terminate  8  years  thereafter. 

2.  At  periodic  intervals,  but  not  less  than  once  each  year,  after  delivery  of  the 
first  unit  to  Buyer,  Buyer  shall  report  its  recorded  maintenance  labor  man¬ 
hour  costs  and  maintenance  materials  costs  for  Manufacturer’s  Equip¬ 
ment.  In  the  event  such  periodic  maintenance  labor  man-hour  costs  or 
maintenance  materials  costs  are  in  excess  of  those  set  forth  in  Attachment 
XXX  hereto,  Manufacturer  agrees  that  it  shall  provide  at  no  additional 
cost  to  Buyer  and  at  the  request  of  buyer,  the  following: 

a.  Technical  service  support  by  qualified  personnel 

b.  Corrective  engineering  design  changes 

c.  Modification  of  all  applicable  Equipment  covered  by  this  Agreement 

d.  Reimbursement  to  Buyer  for  all  costs  incurred,  above  those  sot  forth 
herein,  on  an  annual  basis 

Such  service  as  outlined  above  shall  be  performed  by  Manufacturer  at  Buy¬ 
er’s  request  in  order  to  bring  the  direct  maintenance  labor  man-hours  and 
maintenance  material  costs  within  the  Maintenance  Cost  Guarantees  set 
forth  in  Attachment  XXX. 

3.  This  program  pertains  only  to  maintenance  labor  and  material  costs  for 
Manufacturer’s  Equipment.  All  labor  and  material  costs  shall 
exclude  acts  of  omission  of  Buyer,  acts  of  God  or  a  third  party,  Buyer 
modifications  unrelated  to  improvements  in  operating  cost  of  Manufactur¬ 
er’s  Equipment,  consumable  fluids,  or  outside  maintenance  burden  and 
profit  charges. 

GOVERNMENT  STUDY  RECOMMENDATIONS  PERTAINING  TO 
WARRANTY  USE 

The  following  recommendations  are  excerpted  from  appendix  B,  and  serve  to 
summarize  this  section. 

1.  Use  long-term  contractor  maintenance  warranties  to  motivate  the  contrac¬ 
tor  to  design  for  minimum  life-cycle  cost. 

2.  To  overcome  the  potential  problem  of  spare-parts  stocking  and  field  repair 
of  multiple  equipment  configuration,  make  use  of  depot  repair  or  supplier 
maintenance  under  warranty.  In  the  field,  replace  rather  than  repair  failed 
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replaceable  units.  Include  warranty  requirements  when  initiating  develop¬ 
ment. 

3.  Institute  a  cost  accounting  system  that  will  afford  visibility  of  the  mainte¬ 
nance  process  and  make  possible  realistic  cost  comparisons  between  mili¬ 
tary  and  industrial  maintenance. 

4.  Establish  alternative  sources  of  maintenance,  including  the  maximum  fea¬ 
sible  amount  of  contractor  maintenance,  foster  competition  and  resultant 
efficiency  in  the  maintenance  process  and  to  ensure  the  proper  utilization 
of  scarce  military  personnel  in  the  present  zero-draft  environment. 

5.  Extend  the  application  of  long-term  contractor  maintenance  warranties  to 
military  electronic  procurements. 

6.  Make  known  the  intention  to  contract  for  maintenance  warranties  on  pro¬ 
duction  equipment  at  the  time  development  is  initiated,  so  that  the  con¬ 
tractor  will  design  to  minimize  total  costs  of  production  and  warranty 
maintenance. 

7.  Establish  a  warranty  review  group  within  OSD  to  monitor  results  of  trial 
applications,  to  determine  desirable  warranty  contractual  formats,  and  to 
refine  the  categories  of  equipments  which  warranties  are  most  applicable 
and  for  which  warranties  are  most  effective. 

8.  Initially,  apply  long-term  contractor  maintenance  warranties  to  equip¬ 
ments  whose  failed  units  can  be  replaced  in  the  field  and  conveniently 
returned  to  the  contractor’s  plant  or  base  for  repair,  or  to  which  the  con¬ 
tractor  can  have  ready  access  for  field  repair,  such  as  airborne  communica¬ 
tions,  navigation,  and  identification  equipment;  modular  radars;  vehi¬ 
cular  communication  sets;  complex  manpack  equipment  such  as  LORAN 
C/D;  forward-looking  infrared  (FLIR)  systems;  and  domestic  communica¬ 
tion,  data  processing,  and  radar  installations. 

9.  The  government  should  defer  invocation  of  the  final  product  baseline,  as 
applicable  to  electronic  equipment,  until  field  reliability  objectives  have 
been  achieved,  or,  in  the  case  of  equipment  under  contract  maintenance 
warranty,  until  the  warranty  period  is  about  to  end  and  the  government  is 
about  to  take  over  maintenance  from  the  warrantor. 

10.  Use  contractor  warranties  and  maintenance  to  reduce  the  need  for  techni¬ 
cal  and  maintenance  manuals  and  provisioning  data. 


FINAL  CONSIDERATIONS 


The  use  of  long-term  warranties  is  subject  to  the  same  tradeoff  considerations 
that  apply  to  other  forms  of  nonorganic  maintenance.  Only  in  rare  circumstances  can 
organic  maintenance  be  performed  without  compromising  the  warranty.  Further¬ 
more,  the  use  of  the  equipment  must  lend  itself  to  the  collection  of  valid  field  failure 
data  and  to  a  reasonable  description  of  the  mission  and  use  environments.  In  short, 
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the  mission  must  be  of  a  character  that  allows  nonorganic  maintenance  with  reason¬ 
able  turnaround  time.  It  should  be  noted  that  nonorganic  maintenance  will  generally 
require  higher  procurement  quantities  to  fill  supply  pipelines  and  equipment  pools, 
for  mission-essential  equipments,  these  pools  should  include  an  inventory  hedge 
against  possible  disruptions  such  as  strikes. 
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XVI.  TYPES  OF  CONTRACTS 


TWO  PRINCIPAL  TYPES  OF  CONTRACTS 


Basically,  there  are  two  types  of  contracts  —  fixed  price  and  cost.  The  major 
distinction  between  the  two  is  in  the  nature  of  the  seller’s  obligation.  Under  a  fixed- 
price  contract,  the  contractor  must  produce  the  required  items  or  perform  the  serv¬ 
ices  for  the  firm  fixed  price  or  within  the  ceiling  price  of  an  incentive  contract  or  he 
is  subject  to  the  penalties  provided  in  a  Default  clause.  There  are  various  types  of 
fixed-price  contracts  —  firm  fixed  price  (FFP),  fixed  price  with  escalation  (FPE),  and 
fixed-price  incentive  (FPI). 

The  second  general  category  of  contracts  is  cost  reimbursement.  Under  a  cost- 
type  contract,  the  product  is  not  paid  for  on  the  basis  of  an  invoice  price;  rather  the 
government  pays  tho  contractor’s  cost  of  material  and  labor  and  a  portion  of  his  over¬ 
head  costs  as  provided  in  Cost  Principles  cited  in  the  contract.  Cost-type  contracts 
include  cost,  cost  plus  fixed  fee,  cost  plus  incentive  fee,  and  cost  plus  award  fee. 

Under  a  cost-type  contract,  the  contractor  agrees  to  use  his  best  efforts  to 
complete  the  contract  within  the  estimated  amount  provided  in  the  contract  but  has 
no  obligation  for  further  performance  when,  despite  his  best  efforts,  the  contract  is 
not  fully  porformod  at  the  time  he  expends  the  funds  in  the  contract,  unless  the  Con¬ 
tracting  Officer  increases  the  funds. 

A  time  and  material  type  contract  is  a  hybrid  form  under  which  the  contractor 
is  paid  a  fixed  price  for  each  labor  hour,  which  includes  the  labor  costs,  indirect 
expenses,  and  profit,  plus  material  at  cost.  See  table  XVI- 1  for  a  summary  of  contract 
types. 


Appendix  A  of  this  chapter  is  a  glossary  of  procurement  terms. 


BASIS  OF  SELECTION  OF  THE  TYPE  OF  CONTRACT 


The  mtyor  consideration  in  the  selection  of  the  type  of  contract  should  be 
whether  or  not  the  item  can  be  made  or  tho  services  can  be  performed.  From  the  con¬ 
tractor’s  viewpoint,  if  he  is  sure  that  he  can  make  the  item  and  can  secure  a  price 
which  will  adequately  cover  contingencies  and  risks  in  the  contract,  then  a  firm  fixed- 
price  contract  is  best.  On  the  other  hand,  alter  determining  that  the  contractor  can 
reasonably  be  expected  to  perform  tho  contract,  the  Contracting  Officer  must  deter¬ 
mine  whether  the  price  asked  is  a  reasonable  one  considering  the  risks  in  the  pro¬ 
curement.  If  the  Contracting  Officer  finds  that  there  are  unusual  contingencies 
present  which  the  contractor  has  taken  into  consideration  in  establishing  his  price, 
then  the  Contracting  Officer  must  negotiate  a  price  which  equitably  shares  the  risk 
between  the  contractor  and  the  government,  or  he  should  include  provisions  for  esca 
lation,  with  or  without  incentive  provision,  which  will  allow  him  to  recapture  the 
contingencies  included  in  the  contractor’s  price  if  they  do  not  occur. 
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Table  XVI- 1  Contract  types  (continued) 
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Where  the  contractor  is  not  certain  that  he  can  perform  the  contract  due  to 
uncertainties  in  the  contract  requirements  which  prevent  him  from  developing  a  rea¬ 
sonable  estimate  of  the  cost  of  performance,  or  where  the  contract  calls  for  advances 
in  the  state  of  the  art  which  the  contractor  is  not  sure  he  can  achieve,  then  he  should 
endeavor  to  secure  a  cost-type  contract.  On  the  other  hand,  under  a  cost-type  con¬ 
tract,  the  Contracting  Officer  has  no  assurance  of  the  success  of  work  being  per¬ 
formed  and  no  idea  of  how  much  it  will  eventually  cost.  This  type  of  contract  can  only 
be  used  where  the  Contracting  Officer  determines  that  it  is  likely  to  be  less  costly 
than  other  methods,  or  that  it  is  impracticable  to  secure  supplies  or  services  of  the 
kind  or  quality  required  without  the  use  of  such  types  of  contracts. 

If  he  has  the  option,  the  contractor  should  base  selection  of  the  type  of  con¬ 
tract  on  the  knowledge  acquired  during  the  preparation  of  the  cost  estimate  associ¬ 
ated  with  the  proposal.  The  government  makes  its  decision  on  the  basis  of  com¬ 
petitive  prices  or  the  cost  or  price  analysis  of  the  contractor’s  proposal.  In  addition,  a 
mqjor  factor  in  determining  the  type  of  contract  and  its  terms  is  the  relative  bargain¬ 
ing  strength  and  negotiating  ability  of  both  sides  With  reference  to  this,  FAR  16, 
Types  of  Contracts,  provides  as  follows: 

To  provide  the  flexibility  needed  in  the  purchase  of  the  large  variety 
and  volume  of  military  supplies  and  services,  a  wide  selection  of  types 
of  contracts  is  available  to  the  contracting  parties.  The  respective  con¬ 
tract  types  vary  as  to  (i)  the  degree  and  timing  of  responsibility 
assumed  by  the  contractor  for  the  costs  of  performance,  and  (ii)  the 
amount  and  type  of  profit  incentive  offered  the  contractor  to  achieve  or 
exceed  specified  standards  or  goals  With  regard  to  degree  of  cost 
responsibility,  the  various  types  of  contracts  may  be  arranged  in  order 
of  decreasing  contractor  responsibility  for  the  costs  of  performance.  At 
one  end  is  the  firm  fixed-price  contract  under  which  the  parries  agree 
that  the  contractor  assumes  full  cost  responsibility.  At  the  other  end  of 
this  range  is  the  cost-plus-a-fixed-fee  contract  where  profit,  rather  than 
price,  is  fixed  and  the  contractor's  cost  responsibility  is  therefore  mini¬ 
mal.  In  between  arc  the  various  incentive  contracts  which  provide  for 
varying  degrees  of  contractor  cost  responsibility,  depending  upon  the 
degree  of  uncertainty  involved  in  contract  performance. 

The  purpose  of  the  contractor’s  estimating  process  and  the  government’s  price 
and  cost  analysis  is  to  develop  some  idea  of  the  possible  range  of  costs  within  which 
the  contract  work  may  be  performed.  The  size  of  this  range  from  minimum  to  maxi¬ 
mum  factored  by  the  bargaining  position  will  determine  tho  type  of  contract  used  and 
therefore  the  relative  cost  risk  of  tho  contract  to  both  parties  For  example: 

{  100,000 

“Probable"  Cost 

{95,000  *105,000 

{95,000  {120,000 

{80,000  {140,000 

~ 7  7  7  5  7  7  7  7  7  7  7  7 
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Establishing  the  range  of  possible  cost  is  not  easy.  Since  the  estimate  is  just 
that,  an  estimate,  conclusions  regarding  its  accuracy  can  be  no  more  than  similar 
estimates.  Consideration  of  the  following  factors  will  provide  guidelines  as  to  the 
nature  of  the  type  of  contract. 

1.  Nature  of  the  work.  In  general,  a  high  ratio  of  development  to  fabrication 
will  tend  to  create  a  high  degree  of  uncertainty. 

2.  Past  experience  In  situations  where  both  the  contractor  and  the  govern¬ 
ment  have  had  little  experience  in  estimating  costs  on  similar  work,  confi¬ 
dence  in  the  estimate  will  be  low. 

3.  Time  available.  The  degree  of  confidence  in  the  estimate  increases  substan¬ 
tially  if  adequate  time  is  available  for  careful  estimation. 

Usually,  the  request  for  proposal  will  specify  the  type  of  contract  which  the 
customer  wishes  In  the  majority  of  cases,  however,  proposals  on  alternative  types  of 
contracts  will  be  considered.  Where  the  alternate  proposal  decreases  the  contractor's 
share  of  the  risk  —  for  example,  proposing  on  a  cost-type  basis  when  the  customer 
has  requested  a  fixed-price  proposal  —  it  may  seriously  affect  the  contractor’s 
chances  of  winning.  Following  is  a  brief  description  of  types  of  contracts 

FIRM  FIXED-PRICE  CONTRACT 


This  is  the  basic  type  of  contract  In  formally  advertised  procurements,  the 
firm  fixed-price  contract,  with  or  without  provision  for  escalation,  is  the  only  type 
that  may  be  used.  In  negotiated  procurements,  FAR  directs  that  it  be  used  unless,  un¬ 
der  the  attendant  circumstances,  use  of  another  type  is  more  appropriate 

The  firm  fixed-price  contract,  as  the  name  implies,  is  an  agreement  by  the  con¬ 
tractor  to  furnish  designated  supplies  or  services  at  a  specified  price  which  is  not  sub¬ 
ject  to  adjustment  in  the  light  of  performance  costs.  In  its  basic  form,  the  firm 
fixed-price  contract  carries  the  greatest  risk  and  offers  the  greatest  possibility  of 
profit  or  loss  of  any  type  of  contract.  The  contractor  cannot  collect  more  than  the 
agreed  fixed  price  but  is  entitled  to  receive  the  full  amount  of  tho  fixed  price,  regard¬ 
less  of  his  actual  performance  costs.  This  type  of  contract  is  best  suited  for  procure¬ 
ments  where  reasonably  definite  specifications  are  available,  price  competition  exists, 
production  experience  is  present,  and  costs  can  be  oredicted  with  reasonable 
certainty  Examples  of  items  for  which  this  contract  is  used  are  standard  commercial 
items,  modified  commercial  items,  or  military  items  for  which  adequate  information 
on  production  and  cost  is  available. 

Since  this  type  of  contract  provides  fundamentally  for  a  simple  exchange  of  a 
specified  sum  of  money  for  a  specified  item,  it  is  the  easiest  and  least  costly  of  all  con 
tract  types  to  administer 

Firm  fixed-price  contracts  can  be  used  for  research  or  development  work  when 
(1)  there  is  a  proper  scope  of  work,  (2)  the  contractor  assumes  the  risk  on  a  cost  shar¬ 
ing  basis,  or  (3)  the  contractor  recognizes  tho  risk  and  is  willing  and  financially  able 
to  take  it. 
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FIXED  PRICE  WITH  ESCALATION 


The  fixed-price  contract  with  escalation  provides  for  the  upward  and  down¬ 
ward  revision  of  the  proposed  price  upon  the  occurrence  of  certain  contingencies 
which  are  specifically  defined  in  the  contract. 

Price  escalation  provides  for  an  adjustment  of  the  contract  price  on  the  basis 
of  increases  or  decreases  from  an  agreed-upon  level  in  published  or  established  prices 
of  specific  items  or  in  price  levels  of  the  contract  end  item 

The  use  of  this  type  of  contract  is  appropriate  where  serious  doubt  exists  as  to 
the  stability  of  the  market  and  labor  condition  which  will  exist  during  an  extended 
period  of  production,  and  where  contingencies  which  would  otherwise  be  included  in 
a  firm  fixed-price  contract  are  identifiable  and  can  be  covered  separately  by  escala¬ 
tion  This  type  of  contract  is  difficult  to  administer. 

Escalation  will  usually  be  restricted  to  industry-wide  contingencies  and  labor 
and  material  escalation  will  be  limited  to  contingencies  beyond  the  normal  control  of 
the  contractor  This  type  of  contract  is  limited  to  long-term  production  contracts  for 
standard  items 

Labor  and  material  escalation  provides  for  the  adjustment  of  the  contract 
price  on  the  basis  of  increases  or  decreases  from  agreed  standards  or  indices  in  wage 
rates,  specific  material  costs,  or  both 


FIXED-PRICE  INCENTIVE  CONTRACTS 


The  fixed-price  cost  incentive  contract  is  a  fixed-price-type  contract  with  pro¬ 
vision  for  adjustment  of  profit  and  establishment  of  the  final  contract  price  by  a  for¬ 
mula  based  on  a  relationship  which  final  negotiated  total  costs  bear  to  total  target 
costs  An  incentive  contract  includes  a  target  cost,  a  target  profit,  a  price  ceiling  (but 
not  a  profit  ceiling  or  floor),  and  a  formula  for  establishing  final  profit  and  price 
After  performance  of  the  contract,  the  final  price  is  negotiated  and  the  final  contract 
price  is  then  established  in  accordance  with  the  formula 

Fixed-price  incentive  contracts  are  appropriate  when,  due  to  the  nature  of  the 
work  required,  neither  the  contractor  nor  the  government  has  the  confidence  to  nego¬ 
tiate  a  firm  fixed  price,  but  the  contractor  is  willing  to  take  the  risk  at  the  ceiling 
price  established. 


COST-TYPE  CONTRACTS 


The  cost-rennbursement-type  contract  provides  for  payment  to  the  contractor 
of  allowable  costs  incurred  in  the  performance  of  the  contract  to  the  extent  prescribed 
in  the  contract  This  type  of  contract  establishes  an  estimate  of  total  cost  (1)  for  tne 
purpo&e  of  obligation  of  funds,  and  (2)  to  establish  a  ceiling  which  the  contractor  mu> 
nut  exceed  except  at  his  own  risk  without  prior  approval  or  subsequent  rectification 
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by  the  Contracting  Officer  A  cost-reimbursement-type  contract  is  considered  suitable 
for  use  only  when  the  uncertainties  involved  in  contract  performance  are  of  such  a 
magnitude  that  cost  of  performance  cannot  be  estimated  with  sufficient  reasonable¬ 
ness  to  permit  use  of  any  type  of  fixed-price  type  contract.  In  addition,  it  is  essential 
that  the  contractor’s  cost  accounting  system  be  adequate  for  the  determination  of 
costs  applicable  to  the  contract.  The  contract  normally  provides  for  surveillance  by 
the  buyer  to  ensure  that  inefficient  and  wasteful  methods  are  not  used.  There  are 
various  types  of  cost-type  contracts  —  cost,  cost  sharing,  cost  plus  fixed  fee,  cost  plus 
incentive  fee,  and  cost  plus  award  fee. 

COST  CONTRACT 


The  cost  contract  is  a  cost  reimbursement  type  contract  under  which  the  con¬ 
tractor  receives  no  fee.  Under  this  type  of  contract,  the  government  agrees  to  reim¬ 
burse  the  contractor  for  allowable  costs  of  performance  as  governed  by  FAR  16,  and 
specific  terms  of  the  contract  It  is  used  for  research  and  development  work  with  edu¬ 
cational  institutions  and  other  nonprofit  institutions,  and  for  facilities  contracts 

COST  SHARING  CONTRACT 


The  cost-sharing  contract  is  one  under  which  the  contractor  received  no  fee 
and  is  reimbursed  only  for  an  agreed  portion  of  his  allowable  costs,  as  governed  by 
FAR  16  The  cost-sharing  type  of  contract  recognizes  that  the  contractor  sometimes 
benefits  substantially  (apart  from  profit)  by  the  performance  of  a  government  con¬ 
tract.  This  is  particularly  true  in  the  field  of  development  work,  where  tho  results  of 
the  work  performed  under  a  government  contract  may  have  profitable  commercial 
application.  Where  the  prospect  of  commt.cial  application  can  be  foreseen  at  the  time 
of  entering  into  the  contract,  contracts  are  negotiated  which  provide  that  the  govern¬ 
ment  will  reimburse  the  contractor  for  only  a  specified  percentage  of  its  costs  The 
unreimbursod  portion  of  the  cost  of  performance  represents  the  contractor’s  contribu¬ 
tion  to  what  is,  in  effect,  a  joint  enterprise 

COST-PLUS-A-FIXED-FEE  (CPFF)  CONTRACT 


The  cost-plus-a-fixed-fee  (CPFF)  contract  is  a  cost-reimbursoment-type  con¬ 
tract  which  provides  for  the  payment  of  a  fixed  fee  to  the  contractor.  In  addition,  the 
contractor  is  reimbursed  for  the  allowable  cost  of  performing  the  contract  as  gov¬ 
erned  by  FAR  10,  and  the  terms  of  the  contract 

Present  procurement  law  (10  U.S.C  2306(d))  limits  the  fee  for  performing  a 
CPFF  for  experimental,  developmental,  or  research  work  to  not  more  than  15%  of  the 
estimated  cost  of  the  contract,  not  including  the  fee.  For  architectural  or  engineeung 
services  for  a  public  work  or  utility,  the  fee  is  limited  to  not  more  than  6%  of  the  esti¬ 
mated  cost  of  that  work  or  project,  not  including  the  fie.  The  fee  for  performing  any 
other  CPFF  may  not  bo  more  than  10%  of  the  estimated  cost  of  the  contract,  not 
including  the  fee  Because  the  CPFF  contract  obligates  the  government  to  reimburse 
the  contractor  for  the  allowable  cost  of  performing  the  contract  without  regard  to  the 
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estimated  cost,  it  specifies  a  maximum  amount  beyond  which  the  government  will  not 
be  obligated  to  reimburse  the  contractor.  The  maximum  may  be  more  or  less  than,  or 
equal  to,  the  estimated  cost.  The  contractor  agrees  to  use  his  best  efforts  to  complete 
the  contract  within  the  maximum  limitation,  but  has  no  obligation  for  further  per¬ 
formance  when,  despite  his  best  efforts,  the  contract  is  not  fully  performed  at  the 
time  the  maximum  has  been  reached,  unless  the  Contracting  Officer  increases  the 
maximum. 

Irrespective  of  whether  his  actual  costs  are  greater  or  less  than  the  estimated 
cost,  the  contractor  receives  the  predetermined  fixed  fee.  If  the  scope  of  the  contract 
work  is  increased  or  decreased,  appropriate  increases  or  decreases  both  in  the  esti¬ 
mated  cost  and  the  fixed  fee  are  negotiated. 

There  are  two  forms  of  CPFF  contracts: 

1.  The  Completion  form  is  one  that  describes  the  scope  of  work  to  be  done  as 
a  clearly  defined  task  or  job  with  a  defimto  goal  or  target  expressed  and 
with  a  specific  end  product  required  This  type  is  used  for  the  development 
of  hardware  or  where  a  final  report  of  research  accomplishing  the  goal  or 
target  is  required 

2.  The  Term  form  is  one  which  prescribes  the  scope  of  work  tc  bo  done  in 
general  terms  and  which  obligates  the  contractor  to  devote  a  specified  level 
of  effort  for  a  stated  period  of  time  for  the  conduct  of  research  and  develop¬ 
ment  This  type  of  contract  is  primarily  used  for  basic  research  and  the 
furnishing  of  level-of-effort-type  services 

The  CPFF  contract  is  used  (1)  for  the  performance  of  research,  preliminary 
exploration,  or  study  where  the  level  of  effort  required  is  unknown,  or  (2)  where  the 
contract  is  for  development  and  test  und  the  use  of  cost  plus  incentive  fee  is  not  prac¬ 
tical 

COST-PLUS-INCENTIVE-FEE  (CPIF)  CONTRACT 


Under  this  type  of  contract,  the  government  and  the  contractor  agree  at  the 
time  of  negotiation  of  the  contract  upon  the  target  cost  of  performance  The  target 
fee  is  then  determined  in  relation  to  the  target  cost  Also  established  are  minimum 
and  maximum  fees  and,  finally,  a  fee  adjustment  formula. 

After  performance  of  a  contract,  the  fee  payable  to  the  contractor  is  deter¬ 
mined  in  accordance  witn  the  formula  The  formula  provides,  within  limits,  for  an 
increase  m  fee  above  target  fee  when  the  total  allowable  costs  are  less  than  target 
costs  Conversely,  it  provides  for  decreases  in  the  fee  below  the  target  fee  when  the 
total  allowable  costs  exceed  the  target  costs 

The  incentive-fee  contract  is  used  where  a  cost-reimbursement-type  contract  is 
necessary  and  where  there  is  a  probability  that  its  use  will  result  in  lower  cosU  to  the- 
government  than  other  forms  of  cost-reimbursement- type  contracts  through  cost-ie- 
duction  incentive  to  the  contractor  Maximum  fees  are  subject  to  the  same  percentage 
limitations  previously  mentioned  under  CPFF  contracts 
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The  CPIF  contract  is  suitable  for  use  primarily  for  development  and  test. 
Where  it  is  highly  probable  that  the  development  is  feasible  and  the  government  gen¬ 
erally  has  determined  its  desired  performance  objectives,  the  CPIF  contract  should  be 
used  in  corgunction  with  performance  incentives  m  the  development  of  major  sys¬ 
tems,  and  in  other  development  programs  where  use  of  the  cost  and  performance  ap¬ 
proach  is  considered  both  desirable  and  administratively  practical 

COST-PLUS-AWARD-FEE  (CPAF)  CONTRACT 

The  cost-plus-award-fee  (CPAF)  contract  is  a  cost-reimbursement-type  con¬ 
tract  which  provides  that  the  contractor’s  variable  fee  will  be  determined  subjectively 
by  designated,  high-level  government  personnel  on  the  basis  of  periodic  after-the-fact 
evaluation  of  the  contractor's  performance.  The  CPAF  contract  has  a  base  (in  some 
cases,  the  base  fee  may  be  “zero")  and  provision  for  the  fee  to  be  adjusted  upward  on 
the  basis  of  the  contractor’s  performance  evaluated  in  accordance  with  broad  criteria 
set  forth  in  the  contract.  The  award  fee  determination  is  the  subject  of  special  checks 
and  balances  which  provide  piocedural  safeguards  protecting  the  contractor  from 
arbitrary  or  capricious  evaluations,  but  it  is  not  subject  to  the  Disputes  clause  procc 
dure  The  broad  criteria  against  which  the  contractor’s  ultimate  performance  is 
evaluated  include  performance  of  operations,  technical  management,  business  man¬ 
agement,  and  utilization  of  resources. 

CPAF  contracts  are  considered  appropriate  for  support  services  generally  asso¬ 
ciated  base  maintenance  and  operations  and  mission  support  contracts  In  addition, 
they  are  used  for  contracts  for  the  operation  and  maintenance  of  computer  centers 

TIME  AND  MATERIALS  CONTRACT 


The  time  and  materials  type  of  contract  provides  for  the  procurement  of  sup¬ 
plies  or  services  on  the  basis  of  payment  for  direct  labor  hours  at  specified  fixed 
hourly  rates  (which  rates  include  direct  and  indirect  labor,  overhead,  and  profit)  anil 
materials  at  cost  Material  handling  costs  may  be  included  in  the  charge  for  “material 
at  cost,"  provided  they  are  clearly  excluded  from  any  factor  of  the  charge  computed 
against  diroct  labor  hours  Under  this  type  of  contract,  a  price  ceiling  is  established 
winch  the  contractor  may  not  exceed,  except  at  his  own  risk 

The  time  and  materials  contract  is  used  only  in  those  situations  where  it  is 
not  possible  at  the  time  of  placing  the  contract  to  estimate  the  extent  or  duration  of 
the  work  or  to  anticipate  costs  with  any  substantial  accuracy  Its  use  is  restricted,  us 
its  disadvantage  is  obvious,  since  it  provides  for  payment  of  a  fixed  price  per  applica¬ 
ble  unit  of  time,  it  is  evident  that,  unless  the  rate  is  insufficient  to  cover  the  contrac¬ 
tor’s  cost,,  the  total  amount  of  profit  under  the  contract  is  increased  proportionately 
as  the  number  of  hours  are  increased.  For  this  reason,  the  time  and  materials  con¬ 
tract  is  not  preferred  and  is  used  oidy  after  the  Contracting  Officer  has  determined 
that  it  most  suitably  serves  the  requirement 

This  type  of  contract  is  usually  used  for  (1)  lepair,  maintenance,  or  overhaul 
work,  and  work  to  be  performed  in  emergency  situations,  (2)  engineering  design  and 
preparation  and  revision  of  drawings,  and  (3)  manufacture  of  dies,  jigs,  and  fixtures 
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LETTER  CONTRACTS 


A  letter  contract  is  a  written  preliminary  contractual  instrument  which 
authorizes  immediate  commencement  of  manufacture  of  supplies  or  performance  of 
services,  including,  but  not  limited  to,  preproduction  planning  and  the  procurement 
of  necessary  materials.  It  is  used  when  negotiation  of  a  definitive  contract  in 
sufficient  time  to  meet  the  procurement  need  is  not  possible,  as,  for  example,  when 
the  nature  of  the  work  involved  prevents  the  preparation  of  definitive  requirements, 
specifications,  or  cost  data. 


PROCUREMENT  PLANNING 


Large  contracts  require  an  Acquisition  Plan  (AP),  which  must  be  approved 
through  NAVSUP  prior  to  soliciting  proposals  Smaller  contracts  should  be  carefully 
planned,  too,  to  determine  how  the  contract  should  be  constructed  to  fit  into  the 
overall  system  acquisition  strategy.  The  PM/SE  should  start  coordinating  early  with 
the  contracting  officer  to  assure  thut  all  the  required  elements  are  included.  Coordi¬ 
nate  the  CDRL  through  the  Data  Management  Office  so  that  the  data  requirements 
are  consistent  with  the  contractual  requirements  and  the  acquisition  strategy 
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APPENDIX  A:  GLOSSARY  OF  PROCUREMENT  TERMS 

Term 

Procurement  Usage 

1  Administrative  Contracting 

i  Officer  (ACO) 

A  government  contracting  officer,  often  at  an  in¬ 
stallation  other  than  the  one  which  made  the  con¬ 
tract,  who  handles  the  business  administration  of 
the  contract.  For  the  larger  primes,  the  ACO  is 
commonly  resident  at  the  prime’s  facility. 

Bid 

A  prospective  contractor’s  (bidder’s)  reply  to  a 
formally  advertised  solicitation  document  (IFB). 

Needs  only  government  acceptance  to  constitute  a 
binding  contract. 

j  Bidders  Conference 

In  formally  advertised  procurements,  a  meeting 
of  prospective  bidders  arranged  by  the  contract¬ 
ing  officer  during  the  solicitation  period  to  help 
solicited  firms  fully  understand  the  government’s 
requirement  and  to  give  them  an  opportunity  to 
ask  questions.  (For  research  and  development 
procurements,  see  Presolicitation  Conference.) 

Bidders  (Mailing)  List 
(Master  Bidders  List) 

List  of  sources  maintained  by  the  procuring  office 
from  which  bids  (formal  advertising)  or  proposals 
or  quotations  (negotiation)  can  be  solicited. 

Blanket  Purchase  Agreement 

A  negotiated  contractor  agreement  between  u 
contractor  and  the  government  under  which  indi¬ 
vidual  purchase  orders  not  exceeding  $2500  may 
bo  placed  for  a  specified  period  of  time  and  within 
a  stipulated  aggregate  amount 

“Boder  Plate” 

See  General  Provisions. 

Buy  American  Act 

Federal  statute  imposing  restrictions  on  placing 
contracts  with  manufacturers  who  would  deliver 
items  not  substantially  produced  in  the  United 

States. 

CDRL 

Contract  Data  Requirements  List  (DD  1423) 

Change  Order 

Unilateral  direction  to  a  contractor  to  modify  a 
contractual  requirement  within  the  scope  of  the 
contract,  pursuant  to  the  Changes  clause  con¬ 
tained  in  the  contract.  See  also  Modification  and 

Supplemental  Agreement. 

Commerce  Business  Daily 

See  Department  of  Commerce,  Commerce  Busi¬ 
ness  Daily. 
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Procurement  Usage 


Contract  Type 


-  Basic  Agreement 


Cost-Reimbursement 

Contracts 


-  Fixed-Price  Contracts 


-  Indefinite  Delivery 


-  Letter 


Normally,  a  reference  to  the  pricing  terms  of  the 
agreement  between  a  buyer  and  a  seller,  but  may 
refer  to  the  special  nature  of  other  important 
terms  in  the  agreement.  Thus,  a  contract  may  be 
a  “fixed  price'  type.  Further,  a  “letter  contract” 
may  be  either  a  fixed-price  type  or  a  cost- 
reimbursement  type. 

A  written  instrument  of  understanding  (not  a 
legally  enforceable  contract  per  se)  between  a 
contractor(s)  and  the  government.  Sets  forth  the 
contract  clauses  applicable  to  future  procure¬ 
ments  entered  into  between  the  parties  during 
the  term  of  the  basic  agreement.  Used  to  elimi¬ 
nate  extensive  and  costly  negotiation  when  a  sub¬ 
stantial  number  of  separate  contracts  may  be 
entered  into  with  a  contractor  over  a  definite 
period  of  time. 

In  general,  a  category  of  contracts  whose  use  is 
based  on  payment  by  the  government  to  a  con¬ 
tractor  of  allowable  costs  as  prescribed  by  the 
contract.  Normally  only  “best  efforts”  of  the  con¬ 
tractor  are  involved.  Includes  (i)  cost,  (11)  cost 
sharing,  (iii)  cost-plus-fixed-fee,  and  (iv)  cost- 
plus-incentivo-fee  contracts. 

In  general,  a  category  of  contracts  whose  use  is 
based  on  the  establishment  of  a  firm  price  to  com¬ 
plete  the  required  work.  Includes  (i)  firm  fixed 
price,  (li)  fixed-price  with  escalation,  (iii)  fixed 
price  redeterminablo,  and  (iv)  fixed-price  with  in¬ 
centive  provisions  contracts. 

Used  when  the  precise  quantity  of  items  or  spe¬ 
cific  time  of  delivery  desired  is  not  known.  Usu¬ 
ally  will  specify  a  maximum  and/or  minimum 
quantity.  Such  procurement  is  effected  via  (i)  a 
definite  quantity  contract,  (li)  a  requirements 
contract,  or  (in)  an  indefinite  quantity  contract 
May  be  either  negotiated  or  formally  advertised 

An  into,  im  type  of  contractual  agieement,  some¬ 
times  called  a  “Letter  of  Intent,”  authorizing  the 
commencement  of  manufi  dure  of  supplies  or  per¬ 
formance  of  services.  Used  in  negotiated  procure¬ 
ments  only  when  a  definitixcd  fixed-price  or  cost- 
reimbursement  contract  cannot  be  written  until  a 
later  date. 
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Tinn  Procurement  Usage 

-  Special-Purpose  Contracts  In  general,  a  category  of  contracts  designed  to 

facilitate  a  procurement  for  which  one  of  the 
fixed-price  or  cost-reimbursement-type  contracts 
is  considered  inappropriate.  Includes  (i)  basic 
agreements,  (ii)  indefinite  delivery  type  contracts, 
(iii)  letter  contracts,  and  (iv)  time  and  materials/  ’ 
labor  hour  contracts. 


-  Time  and  Materials/Labor 
Hour  Contracts 


-  Cost  Overrun 


Data 

DAR 


Negotiated  contracts  based  on  specified  fixed 
hourly  rates  to  complete  a  given  task  Used  only 
in  situations  where  it  is  not  possible  at  the  outset 
to  estimate  the  extent  or  duration  of  the  work 
involved  or  to  anticipate  cost  with  any  substan¬ 
tial  accuracy. 

The  amount  by  which  a  contractor  exceeds  (i)  the 
estimated  cost  and/or  (ii)  the  final  limitation 
(ceiling)  of  his  contract. 

All  rocorded  information  to  bo  delivered  under  a 
contract.  “Technical  data’  excludes  management 
and  financial  data. 

Dofeoso  Acquisition  Regulations.  The  implemen¬ 
tation  of  PAR  for  DoD. 


Dofonse  Contract  Administra¬ 
tion  Service  (DCAS) 


Determination  and  Findings 
(D&F) 


Extras 


An  agency,  under  direction  of  Director  of  DSA, 
created  ns  result  of  Project  60  to  provido  unified 
contract  administration  services  to  DoD  compo¬ 
nents  and  NASA,  for  all  contracts  except  those 
specifically  exempted. 

Written  justification  by  a  contracting  officer  or 
higher  authority  for  (i)  entering  into  contracts  by 
negotiation,  (ii)  making  advance  payments  in 
negotiated  procurements,  and  (iii)  determining 
the  typo  of  contract  to  uso. 

Additions  to  items  being  procured,  or  any  quan¬ 
tity  above  that  called  for  by  the  contract  (besides 
allowable  variation  in  quantity),  or  any  combina¬ 
tion  of  these  two 


Federal  .Acquisition  Regulations  (Replaces  ASPR).  The  basic  federal  procurement 

regulations. 
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Term 

Formal  Advertising 


-Bid 


-  Bidders  Conference 


-  Invitation  for  Bids  (IFB) 


-  Request  for  Technical 
Proposal  (RTP) 

-  Responsive  and  Responsible 
Bidder 


-  Two-Step  Advertising 


Genoral  Accounting  Office 
(GAO) 


General  Provisions 


Procurement. 

The  preferred  method  for  government  procure¬ 
ment  of  supplies  and  services.  After  public  open¬ 
ing  of  sealed  competitive  bids,  award  is  made  to 
the  lowest  responsive  and  responsible  bidder 
P™d“ther  factors  considered,  in  accordance 

with  PAT?  1C 


A  prospective  contractor’s  (bidder’s)  reply  to  the 
solicitation  form  used  for  formally  advertised  pro¬ 
curements  (IFB).  Needs  only  the  government’s 
acceptance  for  award  to  be  made. 

A  conference  -  held  after  solicitation,  but  before 
bid  opening  —  oi  all  those  suppliers  who  were 
invited  to  bid  on  the  procurement  to  discuss 
details  of  the  Invitation  for  Bids. 

The  solicitation  form  used  in  formally  advertised 
procurements  and  in  step  two  of  two-step  adver¬ 
tising  (see  below). 


The  solicitation  form  used  to  request  proposals  in 
the  first  step  of  two-step  advertising. 


A  biddor  is  responsive  if  his  bid  conforms  to  the 
requirements  of  the  IFB,  and  is  responsible  if  he 
has  the  capacity  and  facilities  to  produce  the  sup¬ 
plies  or  render  the  services  being  procured. 

A  method  of  procurement  authorized  by  FAR  14  5 
Stop  ono  consists  in  roo.uesting  technical  pro¬ 
posals  from  prospective  contractors.  Step  two 
consists  in  inviting  bids  under  regular  formal 
advertising  procedures  from  those  firms  that  sub¬ 
mit  acceptable  proposals  under  step  one. 


An  agency  of  the  legislative  branch,  responsible 
solely  to  the  Congress,  which  functions  to  audit 
all  negotiated  government  contracts  and  investi¬ 
gate  all  matters  relating  to  the  receipt,  disburse¬ 
ment  and  application  of  public  funds.  Determines 
whether  public  funds  are  expended  in  accordance 
with  appropriations. 


The  mandatory  (by  law  or  regulation)  clauses  for 
all  DoD  contracts  for  the  type  of  procurement 
involved  -  sometimes  called  ‘boiler  plate.”  The 
clauses  devised  particularly  for  the  procurement 
are  called  the  Special  Provisions. 


XVI-14 


Piecuifiment  .Usage 


Government-owned  property  furnished  to  a  con¬ 
tractor  for  the  performance  of  a  contract.  Defined 
as  (i)  industrial  facilities,  (ii)  material,  (iii)  special 
tooling,  (iv)  special  test  equipment,  (v)  military 
property.  Also  designated  GFM,  Government- 
Furnished  Material,  and  GFE,  Government- 
Furnished  Equipment. 

A  type  of  agreement  describing  the  total  desired 
area  of  contractor  performance  and  breaking  this 
down  into  a  number  of  broadly  defined  tasks.  The 
contractor  is  obligated  to  perform  Task  Orders 
subsequently  issued  by  the  government  under  the 
terms  and  conditions  in  the  Master  Contract. 

Any  formal  revision  of  the  terms  of  a  contract, 
either  within  or  outside  the  scope  of  the  agree¬ 
ment.  Includes  Change  Orders.  See  also  Supple¬ 
mental  Agreement. 

The  method  of  procurement  used  when  one  or 
more  of  the  basic  conditions  incident  to  formal 
advertising  is  absent  and/or  when  there  is  justifi¬ 
cation  under  one  or  more  of  the  many  exceptions 
provided  by  Title  10,  U.S.C.  2304(a).  See  FAR  15, 
and  Negotiation  Authority,  below. 

-  Offer/Proposal/Quotation  A  prospective  contractor’s  response  to  the 

solicitation  form  (RFP/RFQ)  used  for  a  negoti¬ 
ated  procurement. 

-  Preeolicitation  Conference  A  conference  between  government  and  industry 

concerning  technical  problems  or  other  areas  of 
importance  relating  to  the  procurement.  Precedes 
the  solicitation  of  prospective  contractors  for  the 
procurement. 

-  Request  for  Proposal  (RFP)  The  solicitation  form  used  for  negotiated  procure¬ 

ments  when  the  government  reserves  the  right  to 
award  without  further  oral  or  written  negotia¬ 
tion.  Only  the  acceptance  of  the  government  is 
required  for  award. 

-  Request  for  Quotation  (RFQ)  The  solicitation  form  used  in  negotiation  to  ob¬ 

tain  price,  cost,  deliveiy,  and  other  information 
from  prospective  suppliers.  Used  when  award  will 
be  made  following  extensive  negotiation  with  the 
offeror.  Award  requires  bilateral  agreement  of 
both  the  contractor  and  the  government. 


Term 

Government-Furnished 
Property  (GFP) 


Master  (Task  Order) 
Contract 


Modification 


Negotiation 
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Negotiation  Authority 


Offer 


Option 


Preaward  Survey  (Facility 
Capability  Review  —  FCR) 


Preproposal  Conference 


Presolicitation  Conference 


Price  and  Fee 
-  Ceiling  Price 


Authority  to  negotiate  a  contract  under  one  of  the 
17  statutory  exceptions  granted  by  Congress, 
rather  than  to  formally  advertise  the  procure¬ 
ment.  FAR  15  describee  circumstances  in  which 
this  authority  may  be  used  and  the  requirements 
for  its  approval. 

A  prospective  contractor’s  response  to  a  solicita¬ 
tion  document  in  a  negotiated  procurement.  The 
contractor  can  be  termed  the  “offeror.”  See  also 
RFP  and  RFQ. 

A  contractual  clause  permitting  an  increase  in  the 
quantity  of  supplies  beyond  that  originally  stipu¬ 
lated  or  an  extension  in  the  time  for  which  serv¬ 
ices  on  a  time  basis  may  be  required. 

Study  of  a  prospective  contractor’s  financial,  or¬ 
ganizational,  and  operational  status  made  prior 
to  contract  award  to  determine  his  responsibility 
and  eligibility  for  government  procurement. 

In  negotiated  procurements,  a  meeting  hold  with 
potential  contractors  a  few  days  after  requests  for 
proposals  have  been  sent  out,  to  promote  uniform 
interpretation  of  work  statements  and  specifica¬ 
tions  by  all  prospective  contractors.  See'also  Bid¬ 
ders  Conference. 

A  meeting  held  with  potential  contractors  prior  to 
a  formal  solicitation,  to  discuss  technical  and 
other  problems  connected  with  a  proposed  pro¬ 
curement.  The  conference  is  also  used  to  elicit  the 
interest  of  prospective  contractors  in  pursuing 
the  task. 


The  negotiated  monetary  limit  —  in  a  fixed-price- 
type  contract  —  to  the  amount  that  the  govern¬ 
ment  is  obligated  to  pay.  Cost  incurred  beyond 
this  point  must  be  absorbed  by  the  contractor. 
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-Fee 


Term 


-  Target  Price 


Procurement  Contracting 
Officer  (PCO) 


Procurement  Request  (PR) 


Procurement  Usage 

An  amount,  in  addition  to  allowable  costs,  paid 
to  contractors  having  CPFF  or  CPIF  contracts. 
In  CPFF  contracts,  the  fee  is  fixed  as  a  percent¬ 
age  (stated  in  a  dollar  amount)  of  the  initially 
estimated  cost  of  the  procurement.  In  CPIF  con¬ 
tracts,  the  fee  is  expressed  in  maximum  and  mini- 
mum  amounts,  along  with  a  fee-adjustment  for¬ 
mula  that  provides  the  incentive  for  a  reduction 
in  cost  to  the  government.  Statutory  limitations 
are  prescribed  for  the  maximum  setting  of  fees. 


The  negotiated  estimate  of  price  —  in  a  fixed- 
price  redeterminable  or  incentive  contract  —  that 
the  government  expects  to  pay  for  supplies  pro¬ 
cured  under  the  contract. 

The  government  contracting  officer  directing  and 
administering  the  procurement  through  the 
award  of  the  contract  and  the  signing  of  the 
actual  contractual  documents.  Administration  of 
the  contract  after  award  may  be  delegated  to  an 
ACO,  as  described  above. 

Document  which  describes  the  required  supplies 
or  services  so  that  a  procurement  can  be  initi¬ 
ated.  Some  procuring  activities  actually  refer  to 
the  document  by  this  title,  others  use  different 
titles,  such  as  Procurement  Directive  and  so 
forth. 


Purchase  Order 


Qualified  Products  List  (QPL) 


Supplemental  Agreement 


A  contractual  procurement  document  used  pri¬ 
marily  to  procure  supplies  and  nonpersonal  serv¬ 
ices  when  aggregate  amount  involved  in  any 
one  transaction  is  relatively  small  (for  example, 
not  exceeding  $25,000). 

A  list  of  products  which  are  pretested  in  advance 
of  actual  procurement  to  determine  which  sup¬ 
pliers  can  comply  properly  with  specification 
requirements.  This  is  most  usually  done  because 
of  the  length  of  time  required  for  test  and  evalu¬ 
ation. 


Bilateral  written  amendment  to  a  contract  by 
which  the  government  and  the  contractor  settle 
price  and/or  performance  adjustments  to  the 
basic  contract.  See  also  Change  Order  and  Modi¬ 
fication. 


XVI-17 


Task  Order 
Termination 


Term  Procurement  UsapiR 

See  Master  Contract. 

The  canceling  of  all  or  a  part  of  a  prime  or  sub¬ 
contract  prior  to  its  completion  through  per¬ 
formance. 
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APPENDIX  B:  STATEMENT  OF  WORK  REQUIREMENTS 


Statement  of  work  (SOW)  writers  should  ensure  that  objectives  and  perform¬ 
ance  requirements  in  the  SOW  are  complete  and  clearly  written  in  conventional  lan¬ 
guage  to  the  extent  practical.  Complete  elimination  of  technical  language  is  not 
required,  but  it  should  be  reduced  to  essentials  Tequired  to  describe  the  task.  The  per¬ 
son  responsible  for  preparing  the  SOW  must  keep  in  mind  that  excess  technical  lan¬ 
guage  or  technical  constraints  can  affect  the  procurement  beyond  simply  stating  or 
directing  the  contract  effort.  It  may  also  affect  the  number  of  good  sources  willing 
and  able  to  respond. 

BRAND  NAME  OR  EQUAL 

Occasionally,  a  requirement  will  be  of  such  nature  that  it  can  be  met  by  one  of 
several  commercial  products.  When  this  situation  exists,  it  is  frequently  possible  to 
make  the  procurement  on  a  “Brand  Name  or  Equal’  basis.  Brand  name  does  not 
require  a  full  statement  of  work,  but  does  oblige  the  requiring  activity  to  specify  all 
the  technical  characteristics  which  are  necessary  to  fulfill  the  requirement.  These 
characteristics  become  the  specification  against  which  “equal  products  can  be  meas¬ 
ured.  A  sample  SOW  for  a  Brand  Name  or  Equal  procurement  appears  at  the  end  of 
this  appendix. 

SOW  CHECKLIST 

The  following  checklist  for  work  statements  provides  some  of  the  considera¬ 
tions  which  the  writer  must  bear  in  mind: 

•  Is  the  SOW  sufficiently  specific  to  permit  the  writer  and  the  contractor  to 
make  a  list  of  manpower  and  resources  needed  to  accomplish  it? 

•  Are  specific  duties  of  the  contractor  stated  in  such  a  way  that  he  knows 
what  is  required  and  the  technical  representative  who  signs  the  acceptance 
report  can  tell  whether  the  contractor  complied? 

•  Are  sentences  written  so  that  there  is  no  question  as  to  whether  the  con¬ 
tractor  is  to  be  obligated  (that  is,  “the  contractor  shall  do  this  work,"  not 
“this  work  will  be  required")? 

•  Is  the  proper  reference  document  shown?  Is  it  really  pertinent  to  the  task? 
Fully  or  partially?  Is  it  properly  cited? 

•  Have  the  elements  of  quality  assurance  been  fully  considered  for  the  total 
life  of  the  procurement  requirement? 

•  Are  any  military  specifications  or  exhibits  applicable?  In  whole  or  in  part? 
If  so,  are  they  properly  cited?  (Use  the  latest  available  revision  or  issue  of 
each  document.) 

•  Is  general  information  separated  from  direction  so  that  background  infor¬ 
mation,  suggested  procedures,  and  the  like  are  clearly  distinguishable  from 
contractor  responsibilities? 
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•  Is  there  a  date  for  each  thing  the  contractor  is  to  do  or  deliver?  If  elapsed 
time  is  used,  does  it  specify  calendar  days  or  work  days? 

•  Have  the  headings  been  checked  for  format  and  grammatical  usage?  Are 
subheadings  comparable?  Is  the  text  compatible  with  the  title?  Is  a  multi- 
decimal  numbering  system  used? 

•  Have  extraneous  material  and  cross  references  to  contract  clauses  and  gen¬ 
eral  provisions  been  expunged? 

•  Are  task/line  item  and  end  item  procurement  provisions  mutually  discrete 
with  regard  to  development  and  test  versus  production  activities? 

•  Have  all  requirements  been  reviewed  to  ensure  compatibility  with  the  data 
requirements  specified  on  DD  Form  1423? 

•  Have  all  extraneous  data  requirements  been  eliminated? 

•  Are  all  obligations  of  the  government  carefully  delineated?  (If  government 
furnished  equipment  is  to  be  provided,  the  nature,  condition,  and  availabil¬ 
ity  of  the  equipment  should  be  stated.  If  approval  actions  are  to  be  made 
by  the  government,  provide  for  a  time  limit.  Remember  that  any  provision 
which  takes  control  of  the  work  away  from  the  contractor,  even  temporar¬ 
ily,  must  be  covered  by  a  contingency  reserve  if  the  contractor  is  to  protect 
himself.) 

•  Have  all  loopholes  been  closed?  (Contractors  and  inspectors  go  by  the  let¬ 
ter  of  the  SOW.  The  contractor  may  refuse  to  do  something  that  is  only 
referred  to,  desired,  or  described  as  a  goal.) 

•  fa  the  requirement  completely  described?  (To  be  legal  and  binding,  an 
agreement  must  be  complete.  Not  only  for  reasons  of  legality,  but  for  every 
practical  application,  it  is  necessary  that  the  details  be  complete.  Specify 
when  and  where  as  well  as  what.) 

•  Since  they  generally  result  either  in  an  expensive  disagreement  or  in  a 
windfall  to  the  contractor,  have  all  “catchall”  statements  been  eliminated? 

•  Does  the  SOW  sole  source  the  work?  (The  SOW  specifies  a  requirement  of 
the  government  and  is  supposedly  impartial  concerning  who  can  do  it.  In 
keeping  with  this  philosophy,  the  work  statement  itself  should  contain  no 
reference  to  sources  or  proprietary  talent.) 

»  Is  the  requirement  overspecified?  (The  ideal  situation  is  to  specify  results 
required  and  let  the  winning  contractor  find  the  best  method  of  getting 
there.) 

•  Has  the  work  been  organized  into  tasks?  (These  are  helpful  in  evaluating 
and  during  performance  may  be  used  for  control.) 

a  Have  all  points  of  control,  where  needed,  been  included  (for  example,  sub¬ 
mission  of  designs  for  approval)? 

e  Does  the  SOW  include  only  such  reports  and  documentation  as  ore  really 
needed  for  control;  documentation  of  technical  results;  and  follow-on 
procurements? 
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FORMAT  OF  THE  STATEMENT  OF  WORK 


The  importance  of  work  statements  is  such  that  consistency  of  format  will 
make  them  much  easier  to  review  and  understand.  A  general  format  and  headings  for 
SOWs  are  as  follows: 

1.0  Introduction  (Objective) 

2.0  Scope 

3.0  General  background  (information,  constraints,  and  reference 
documents) 

4.0  Task/Technical  Requirements 
5.0  Reports,  Data,  and  other  Deliverables 
6.0  Special  Considerations 

Other  formats  can  be  used  or  may  be  directed  by  specific  contract  requirements.  For 
statements  of  work  prepared  in  support  of  SYSCOMS,  use  MIL-HDBK-245. 

STATEMENT  OF  WORK  CONTENT 

The  content  of  the  SOW  is  described  by  paragraph  below. 

1.0  Introduction  (Objective).  The  introduction  is  intended  to  give  a  brief  over¬ 
view  of  the  specialty  area  and  leads  up  to  why  this  particular  new  program  is  being 
pursued.  The  overall  requirement  which  needs  fulfillment,  the  present  difficulties  or 
deficiencies  which  do  not  permit  the  requirement  to  be  met,  and  the  determinations 
which  must  bo  made  to  solve  the  problems  should  be  outlined  briefly,  in  fully  under¬ 
standable  terms.  Quite  often  an  understanding  of  the  value  of  the  technical  objective 
can  be  reinforced  by  inclusion  of  an  explanation  of  the  payoff  that  this  technical 
objective  will  have  to  future  naval  systems  capability.  In  framing  the  objective,  think 
clearly  on  how  the  results  will  be  used.  The  stated  objective  should  be  consistent  with 
the  funds  planned  and/or  with  the  minimum  requirements. 

2.0  Scope.  This  section  provides  an  overall  picture  of  the  desired  work  pro¬ 
gram  in  concise  form.  The  scope  will  outline  the  various  phases  of  the  program  and 
tie  down  the  overall  limits  of  the  program  in  terms  of  specific  technical  objectives, 
time,  and  any  special  provisions  or  limitations  It  must  be  consistent  with  the 
detailed  requirements.  This  section  should  also  describe  in  a  clean-cut  statement  the 
end  result  desired  or  what  tho  “product’  of  the  effort  should  be.  Don’t  overextend  the 
magnitude  expected,  or  an  overrun  may  be  the  result. 

3.0  General  Background.  Include  any  background  information,  explanations, 
or  constraints  which  are  necessary  in  order  to  understand  the  requirements.  Discuss 
how  the  procurement  arose,  indicate  its  relationship  to  previous,  concurrent,  and 
future  operations,  including  the  threat  analysis,  and  relate  details  which  reveal  its 
purpose  and  significance.  Statements  on  the  importance  of  the  new  work  may  also  be 
included.  Techniques  which  have  previously  been  tried  and  found  ineffective  should 
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be  included.  Frequently  it  is  best  to  leave  the  writing  of  the  background  to  the  last. 
The  listing  of  applicable  technical  reports  resulting  from  the  Defense  Technical  Infor¬ 
mation  Center  (DTIC)  bibliography  search  should  be  entered  here.  Any  such  listing  in 
this  paragraph  is  for  information  only  and  is  not  contractually  obligatory.  All  con¬ 
tractually  applicable  documents  must  be  cited  either  in  the  text  of  the  appropriate 
task  or  in  a  separate  paragraph  entitled  “Applicable  Documents  List.”  If  there  are  no 
applicable  reports,  a  comment  to  this  effect  should  be  made  on  the  supplemental  sheet 
to  the  purchase  request  but  not  included  in  the  SOW. 

4.0  Technical  Eequirements/Tasks: 

4.1  Areas  of  Consideration.  This  paragraph  should  define  the  work  to  be 
accomplished  and  indicate  the  main  steps  and  actions  which  are  required  of  the  con¬ 
tractor  to  conduct  the  program  properly.  These  main  steps  constitute  the  work  phases 
(recommended  approach).  The  technical  leadership  provided  by  the  government  in 
planning  and  establishing  the  contractual  program  appears  here.  It  should  not  reflect 
an  attitude  that  this  is  the  only  approach  to  the  problem.  It  should  state  that  this  is  a 
suggested  method  but  new  or  unique  ideas  supported  by  available  data  are  acceptable 
and  encouraged  This  paragraph  also  gives  known  specific  phenomena  methods  which 
could  contribute  to  a  solution,  possible  correlation  with  existing  knowledge,  opera¬ 
tional  and  installation  environments  anticipated  for  the  ultimate  operational  equip¬ 
ment,  and  other  factors,  including  all  available  foreign  technology  information,  such 
as  would  tend  to  assure  that  the  bidder/contractor  would  conduct  a  fully  effective  pro¬ 
gram. 

4.2  If  the  work  encompasses  several  areas  or  lends  itself  to  division  into  tasks, 
this  should  be  indicated.  The  essential  procedures  (that  is,  theoretical  analyses, 
design,  fabrication,  checkout,  tests,  verification,  formulation  of  final  recommenda¬ 
tions,  etc.),  with  limits  on  each,  constitute  the  bulk  of  this  paragraph.  In  some  cases, 
the  engineer  may  wish  to  indicate  the  percent  of  the  total  effort  each  phase  is  to 
receive.  If  there  are  existing  specifications  with  paragraphs  that  define  what  you 
want  to  have  the  contractor  do  in  terms  of  tests,  etc.,  use  them  or  incorporate  by  ref¬ 
erence,  as  appropriate,  rather  than  compose  original  paragraplis.  Specify  those  con¬ 
siderations  which  may  guide  the  contractor  in  his  analysis,  design,  or  experimenta¬ 
tion  on  the  designated  problem.  These  should  include  operational  characteristics  (if 
any)  or  other  factors  the  contractor  is  expected  to  consider  in  performing  under  the 
contract.  Definitions  may  also  be  included  or  can  be  identified  in  a  separate  section. 

4.3  Be  sure  that  limits  of  environment,  test  durations,  combustion  pressures, 
data  recording,  expansion  ratio,  mixture  ratio,  range  of  particle  size,  etc.,  are  speci¬ 
fied.  Criteria  governing  the  number  of  designs,  types  of  propellants,  performance, 
hardware  size,  number  of  tests,  etc.,  and  constraints  such  as  budget,  environmental, 
producibility,  and  risk  levels  should  be  included  in  the  definitization  of  the  work  to 
be  done  by  the  contractor.  This  may  be  better  set  forth  in  a  detail  specification. 

4.4  Commit  yourself.  When  the  burden  of  definition  must  be  placed  on  the 
offeror,  clearly  impose  the  requirement  in  such  a  manner  that  he  understands  that  he 
must  provide  this  definition  in  the  bid  (if  this  >s  what  is  wanted)  or  later  on  in  the 
contractual  program  (if  this  is  the  intent).  Any  specific  limitation  such  as  “not 
desired”  or  “previously  tried”  techniques  should  be  stated.  If  there  is  a  primary  area 
with  a  secondary  contributing  or  limiting  area,  these  should  be  defined.  Experimental 
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or  installation  environments  (known  or  anticipated),  scientific  or  technical  personnel, 
or  other  resources  should  be  indicated.  When  the  bidder  provides  definition  or  plans, 
it  should  be  stipulated  that  these  are  subject  to  Navy  approval. 


4.5  A  description  should  be  given  of  any  end  item  that  is  the  subject  of  devel¬ 
opment.  It  will  firmly  and  clearly  define  the  required  work  for  such  tasks  as  those 
listed  below. 

a.  Review  of  current  literature  to  establish  a  basis  for  further  research, 
analysis,  investigation,  or  experimentation. 

b.  Search  for  new  ideas  through  investigation  of  various  phenomena. 

c.  Paper  or  theoretical  analysis  of  ideas  in  relation  to  requirements,  ultimate 
use,  and  trade-off  capabilities. 

d.  Computational  analysis  and  formulation  of  mathematical  model.  Experi¬ 
mentation  to  evolve  methods  of  instrumentation. 

f.  Derivation  of  a  basic  equipment  design  or  experimental  assemblies. 

g.  Test  and  evaluation. 

4.6  If  the  state  of  the  art  is  such  that  one  or  more  specific  methods  of 
approach  to  the  solution  are  to  be  followed,  this  section  should  indicate  the  desired 
approach.  If  no  specific  approach  is  primarily  warranted  and  one  will  be  determined 
on  the  basis  of  the  selected  contractor’s  technical  proposal,  this  need  not  be  men¬ 
tioned,  and  this  section  should  include  a  statement  of  criteria  on  which  a  choice  of 
alternative  approaches  will  be  based. 

4.7  Scientific  and  Technical  Information.  Insert  the  following,  if  applicable: 
“The  contractor  shall  search  the  existing  sources  of  scientific  and  technical  informa¬ 
tion  to  determine  the  current  state  of  the  art  to  avoid  duplication  of  effort  and  con¬ 
serve  scientific  and  technical  resources.’  Ensure  that  all  generated  scientific  and 
technical  information  that  has  significant  value  to  the  pertinent  scientific  and  techni¬ 
cal  communities  is  furnished  to  the  DT1C. 

5.0  Reports,  Data,  and  Other  Deliverables.  Contract  data  or  reporting  require¬ 
ments  should  not  be  duplicated  in  the  SOW.  DD  Form  1423  is  the  medium  for  estab¬ 
lishing  the  requirements.  The  SOW  may  refer  to  the  DD  Form  1423  incorporated  in 
the  contract  by  reference  or  even  to  any  particular  data  item  for  clarifying  a  require¬ 
ment.  If  deliverable  hardware  is  required,  it  should  also  be  listed  in  this  section  as  a 
separate  paragraph. 

6.0  Special  Considerations.  A  paragraph  outlining  any  special  interrelation¬ 
ships  between  the  contractors  for  use  of  government-furnished  or  loaned  property,  for 
example,  may  be  devised  and  added  to  the  SOW  as  paragraph  6.0.  Any  other  specific 
directions  relative  to  technical  work  (not  administrative  matters)  for  the  contractor 
to  follow  should  be  included  here.  This  paragraph  might  also  provide  instructions  to 
the  contractor  relative  to  the  possible  utilization  of  government  expertise. 
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WORK  PACKAGES 


An  integral  aspect  of  both  understanding  and  developing  a  statement  of  work 
is  the  ability  to  separate  the  complete  effort  into  smaller,  concise  work  packages.  The 
reasons  for  doing  this  are  several: 

The  job  is  much  easier  to  estimate  by  segment  than  as  a  total. 

The  work  packages  essentially  become  the  task  breakouts  in  the  final 
SOW. 

The  prospective  contractors  will  segment  the  work  in  their  responses, 
giving  another  dimension  for  evaluation. 

In  many  instances,  work  packages  can  be  used  to  indicate  a  desired 
approach  to  a  problem. 

The  action  of  generating  work  packages  will  indicate  to  the  technical 
code  areas  in  which  it  is  not  sure  of  the  specifics  of  its  requirements. 

The  breakout  of  work  packages  cannot  be  formalized  into  a  procedure,  since 
each  one  will  be  different  from  all  others  —  just  as  each  SOW  is  different.  The  effort 
can  only  be  dofined  as  a  realistic  separation  of  a  total  effort  into  its  reasonable  parts. 
In  general,  the  work  packages  should  conform  to  the  project  work  breakdown  struc¬ 
ture  (WBS)  (seo  chapter  III,  Program  Planning  —  Organization). 

SOW  TASK  AREAS 


The  content  of  specifications  is  limited  to  technical  requirements  in  terms  of 
design,  performance,  and  test.  The  exactness  of  design  details  may  be  specified  to  the 
extent  necessary  to  ensure  the  interchangeability  of  certain  components,  modules,  or 
parts.  In  view  of  these  limitations,  it  becomes  necessary  to  establish  other  contract 
tasks  in  the  SOW  to  supplement  the  specification.  The  following  task  areas  are  typi¬ 
cal  in  support  of  a  specification: 

1.  Integrated  Logistic  Support  Program  Requirements 

2.  Configuration  Management  Program 

3.  Technical  Manual  and  Publications 

4.  Training  Requirements 

5.  Acquisition  Management  System  Requirements 

6.  Supply  Support  Tasks 

7.  Engineering  Drawings  and  Associated  Lists,  Tasks  and  Ordering  Data 
(paragraph  G.2  of  M1L-D-1000) 

8.  Contractor  Services  Requirements 
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9.  Quality  Program  Requirements 

10.  Level-of-Repair  Program  Requirements 

11.  Test  Point,  Nodes,  Calibration,  and  Instrumentation  Information 
Requirements 

12.  Reliability  Program  Requirements 

13.  Maintainability  Program  Requirements 

14.  Human  Factors  Program  Requirements 

15.  Safety  Program  Requirements 

16.  PMS  (Planned  Maintenance  Subsystems)  Requirements 

17.  Electromagnetic  Compatibility  Program  Requirements 

18.  TEMPEST  Control  Program  Requirements 

In  addition,  thero  are  project  tasks  which  go  beyond  the  scope  of  tho  technical 
effort  and  its  management  which  also  must  be  included  in  the  SOW.  The  following 
task  areas  suggest  some  of  these  “other"  tasks: 

1.  The  development  of  specifications  and  data  for  future  project  phases 

2.  Tasks  in  support  of  a  design-to-cost  effort  or  lifo-cycle  cost  procurement 

3.  Standardization  program 

4.  Value  engineering  program 

5.  Special  research,  investigation,  or  study  tasks  independent  of,  but  support¬ 
ing,  tho  equipment  specified 

6.  Special  data/information  requirements  (which  will  be  ordered  in  the 
CDRL)  not  derived  from  a  specification  requirement  or  SOW  program 
requirement 
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Date 

SAMPLE 
Statement  of  Work 
Brand  Name  or  Equal 

Electric  Motor-Generator  Set,  60  Hz  to  400  Hz 
XYZ  Company,  Model  1234,  or  Equal 


1.0  In  troc  action 

1.1  This  statement  of  work  specifies  a  NOSC  requirement  for  fabrication  and 
delivery  of  a  commercial  electric  motor-generator  set,  60  Hz  to  400  Hz.  The  XYZ 
Company  Model  1234  motor-generator,  or  an  equal,  will  meet  this  requirement. 

2.0  Technical  Requirements 

2.1  The  contractor  shall  provide  all  necessary  materials  and  services  to  fabri¬ 
cate  an  electric  motor-generator  to  meet  the  specified  characteristics  and  limits  in  the 
attached  NOSC  Specification  0004780. 

/sJ _ 
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APPENDIX  C:  EVALUATING  PROPOSALS 


The  purpose  of  the  evaluation  is  to  select  the  sourcefy)  whose  proposal  has  the 
highest  degree  of  realism  and  credibility  and  whose  performance  is  expected  to  best 
meet  the  government  objectives  at  an  affordable  cost.  This  is  to  be  done  in  a  manner 
which  assures  an  impartial,  equitable,  and  comprehensive  evaluation  of  each  competi¬ 
tor’s  proposals  and  related  capabilities  and  which  minimizes  the  complexity  of  the 
solicitation  while  maximizing  the  efficiency  of  the  evaluation/selection  decision.  This 
process  is  directed  by  DoD  Directive  4105.62  of  6  January  1976. 

The  evaluation  process  includes  an  evaluation  plan,  a  solicitation,  the  actual 
evaluation,  negotiations  to  clarify  details,  and  source  selection.  The  evaluation  plan  is 
formulated  prior  to  the  solicitation  and  serves  the  following  purposes: 

To  ensure  that  all  efforts  are  directed  toward  a  common  goal 

To  collect,  organize,  and  display  the  performance,  schedule,  and  cost 
requirements  by  emphasizing  pertinent  evaluation  criteria 

To  provide  a  structure  for  organizing  the  evaluation  group  and  sched¬ 
uling  its  activities 

To  provide  a  structure  for  the  preparation  of  the  RFP 

To  establish  a  format  for  discussion  at  preproposal  conferences,  if  hold, 
and  later  offeror  or  contractor  discussions 

To  serve  as  a  guide  for  tlio  contracting  authority  in  source  selection 

To  provide  procedures  and  methodology  for  evaluation  purposes 

To  provide  guidelines  for  making  tradeoffs  among  and  within  the  vari¬ 
ous  factors  to  the  performance  of  the  equipment  and  to  the  manage¬ 
ment  of  tho  project  in  relationship  to  the  development,  production, 
operating  and  support  costs,  tho  delivery  schedule  and  quantity,  and 
tho  qualitative  requirements  of  tho  procurement 

Tho  solicitation  should  request  proposals  in  two  parts,  the  first  containing  tho  techni¬ 
cal  and  management  responses  and  tho  second  addressing  costs.  Page  limitations  on 
the  solicitation  and  on  the  proposals  are  encouraged,  provided  that  completeness  is 
not  sacrificed.  Tho  solicitation  should  state  technical  goals,  tolerances,  and  acceptable 
values  within  which  tradeoffs  can  be  made.  The  solicitation  also  must  request  the 
types  of  detailed  information  which  will  allow  an  evaluation  of  the  technical 
approach,  the  respondoe’s  understanding  of  the  complexity  of  offort  and  risks,  the 
proposer’s  experience  and  capabilities  to  perform  the  required  tasks,  and  the  realism 
of  schedules  and  costs.  The  solicitation  also  states  tbs  criteria,  methods,  and  proce¬ 
dures  used  to  conduct  the  evaluation,  but  not  the  weighting  and  scoring. 

The  evaluation  plan  should  include  the  following 

•  Evaluation  elements  incorporating  all  specification  and  SOW  requirements 
(each  element  may  address  a  goneral  area  covering  many  individual  specifi¬ 
cations  and  SOW  requirements) 
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•  Element  weighting  based  on  system-critical  requirements  and  the  acquisi¬ 
tion  strategy  (as  a  minimum,  a  qualitative  weighting  of  critical,  important, 
noncritical,  and  desirable  should  bo  used);  a  numeric  weighting  is  pre¬ 
ferred,  especially  when  it  is  derived  from  a  system-effectiveness  model). 

•  Scoring  plan  (a  method  of  combining  raw  scores  from  each  evaluator  for 
each  element  into  a  single  proposal  score) 

•  I'rocedures  for  resolving  inconsistent  scores 

•  Acceptance  criteria 

In  general,  the  scoring  plan  should  assign  a  zero  for  unacceptable  proposals  for  any 
critical  elements.  The  acceptance  criteria  should  identify  the  conditions  under  which 
a  proposal  will  be  accepted  without  modifications,  may  bo  modified  through  negotia¬ 
tions,  or  rejected.  The  project  manager,  system  engineer,  and  contracting  officer 
should  mutually  agree  to  the  plan  prior  to  the  release  of  the  RFP. 

The  actual  evaluation  is  based  upon  the  scoring  of  each  proposal  against  the 
preestablished  evaluation  criteria  ranked  in  order  of  importance  and  weighted  accord¬ 
ingly.  Specific  factors  and  rankings  will  vary  with  each  procurement,  but  the  cost 
proposal  will  always  be  evaluated  separately  from  the  other  part.  In  general,  the  scor¬ 
ing  should  follow  these  guidelines: 


Raw.Sgorfi 

9-10  (90-100) 


7-8  (70-80) 


5-6(50-00) 


0  (0-49) 


Dfisciiplian 

Excellent  —  comprehensive  and  complete; 
meets  or  cxcoeds  nil  proposal  require¬ 
ments;  exemplifies  complete  understanding 
of  the  requirements;  and  demonstrates  in 
detail  how  to  accomplish  the  task. 

flood  —  generally  meots  or  exceeds  pro¬ 
posal  requirements;  omissions  are  of 
minor  consequence  or  small;  would  bo 
likoly  to  produce  an  acceptable  end  item. 

Adequate  —  omissions  are  of  significance, 
but  are  correctable;  substantiation  of 
points  is  weak  or  lacking,  probability  of 
successful  effort  is  marginal. 

Unacceptable  —  gross  omissions;  failure  to 
understand  problem  areas;  failure  to 
respond  to  requirements;  little  or  no 
chance  of  success  in  completing  the  end 
item. 


Each  evaluator  will  formulate  questions  on  each  proposal,  and  the  consolidated  ques¬ 
tions  can  be  submitted  to  the  offeror  for  clarification.  In  each  case,  the  government 
evaluates  each  proposal  against  its  own  previous  estimates  to  assess  the  realism  of 
cost,  schedule,  risk  assessment,  and  technical  approach,  and  against  established  stan¬ 
dards  for  management,  accounting  practices,  and  the  like.  Proposals  which  are  unre- 
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alistic  in  terms  of  technical  and  schedule  commitments  or  unrealistically  low  in  cost 
or  price  can  be  rejected  on  the  grounds  that  the  offeror  fails  to  comprehend  the  com- 
phnahee  and  risks  of  the  contract  requirements  or  else  has  made  an  improvident  pro- 
posahOmyproposals  which  are  evaluated  as  acceptable  (by  preestablished  criteria) 
will  be  passed  for  final  source  selection. 

S?UTCe  ^tion  is  “  inte^ated  decision  based  on  a  consideration  of 
technical  approach,  capability,  management,  design-to-cost,  historical  performance 
P?rtl“ent  factors; Tho  selected  sourcefs)  should  be  the  one(s)  ’ 
who  isex^cted  to  do  the  best  overall  job  for  the  government  at  the  best  price.  The 
selected  offeror  8  proposal  (technical,  management,  and  cost)  must  satisfy  the  Govern¬ 
ment  s  minimum  requirement.  ' 
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XVII.  DEVELOPMENT  ALTERNATIVES 


It  is  desirable  to  avoid  development  —  with  its  associated  risks,  tests, 
and  time  requirements  —  whenever  possible.  Nevertheless,  some  devel¬ 
opment  may  be  necessary  to  support  the  various  phases  of  the  acquisi¬ 
tion  cycle. 

DEVELOPMENT  IN  THE  CONCEPTUAL  PHASE 


The  purpose  of  exploratory  development  is  to  investigate  or  evaluate  the  feasi¬ 
bility  or  practicality  of  a  concept,  device,  circuit,  or  equipment  in  rough  experimental 
breadboard/!) rassboard  form  without  regard  to  the  eventual  overall  fit  or  final  form.  A 
concept  can  frequently  be  proved  in  theory  through  paper  analysis,  simulation  tech¬ 
niques,  or  the  mere  existence  of  similar  commercial  products;  however,  these  tech¬ 
niques  may  have  an  unacceptable  degree  of  uncertainty  attached  to  them  or  may 
restrict  program  options  unnecessarily.  Exploratory  development  as  a  method  of 
proving  feasibility  is  relatively  inexpensive  compared  to  later  development  efforts  and 
may  be  pursued  to  excellent  advantage  if  certain  guidelines  are  followed  to  avoid  pit- 
falls.  In  combination  with  some  degree  of  operations/threat/technology/force  objec¬ 
tives  analysis,  explor»tory  development  can  also  serve  to  validate  the  operational 
requirements  statement. 

The  mqjor  problem  to  be  solved  in  setting  up  an  exploratory  development  pro¬ 
gram  is  to  maintain  options  in  the  follow-on  phases;  all  too  often,  a  course  of  action 
picked  in  pursuing  exploratory  development  will  dictate  the  future  of  the  entire 
acquisition  program.  The  prim  •  alternatives  in  conceptual  phase  development  are 
the  following 

In-house  development 

In-house  development  with  industry  support 

Industry  development  of  technology 

Industry  development  of  the  technical  approach 

The  in-house  alternatives  are  flexible;  when  industry  support  is  required, 
small,  well-defined  work  packages  should  be  formulated.  The  primary  difficulty  with 
in-house  approaches  is  in  creating  and  maintaining  a  diversity  of  possible  technical 
approaches  because  of  the  natural  biases  of  any  single  organization;  nevertheless,  a 
number  of  different  approaches  can  be  generated  quite  efficiently  if  that  requirement 
is  stated  in  the  development  tasking.  When  industry  is  utilized  to  develop  the  techni¬ 
cal  approach,  a  number  of  contractors  should  be  utilized  to  engender  competition 
between  approaches.  The  contracts  should  procure  all  rights  and  documentation  to 
the  developed  approaches;  otherwise,  the  government  may  later  find  itself  in  a  sole- 
source  position  with  the  contractor  dictating  the  prices.  Multisource  contracting 
should  also  be  employed  when  industry  is  used  to  develop  technology;  again,  the  pro¬ 
curement  of  rights  and  documentation  is  important  because  the  high  exploratory 
development  risks  may  yield  only  a  single  successful  technology  which  the  govern¬ 
ment  may  wish  to  exploit  with  multiple  sources  in  the  future. 

In  any  of  the  alternatives,  it  is  important  to  document  and  maintain  some 
configuration  control  on  the  developed  item  so  that  the  knowledge  gained  is  not  lost 
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to  future  phases.  Another  important  consideration  is  the  decision  for  in-house  or 
industry  development  in  any  future  phase  of  the  acquisition  cycle.  If  industry  plays  a 
mqjor  role  in  the  conceptual  phase,  it  is  difficult  to  pursue  any  in-house  development 
later  regardless  of  the  program  decision  factors  because  of  the  political  forces  which 
are  activated  by  industry  participation. 

When  it  is  necessary  to  sole-source  industry  participation  in  exploratory  devel¬ 
opment,  the  participation  should  be  on  only  one  of  several  possible  alternative 
approaches.  Because  of  the  potent  political  forces  accompanying  industry  involve¬ 
ment,  the  industry  tasks  should  be  divorced  from  the  program  in  name  and  funding 
—  that  is,  totally. 

The  system  concepts  and  overall  system  design  are  the  responsibility  of  the 
Government.  The  use  of  industry  during  the  conceptual  phase  does  not  and  should 
not  abrogate  this  responsibility. 


DEVELOPMENT  IN  THE  VALIDATION  PHASE 


Advanced  development  is  intended  to  demonstrate  the  technical  feasibility  of  a 
design,  to  determine  its  ability  to  meet  existing  performance  requirements,  to  secure 
engineering  data  for  use  in  further  development,  and,  where  appropriate,  to  establish 
the  technical  requirements  for  contract  definition.  Dependent  on  the  complexity  of 
the  equipment,  technical  risks,  and  other  technological  factors  involved,  advanced 
development  may  necessarily  be  an  iterative  process  in  order  to  achieve  the  develop¬ 
ment  objectives.  The  final  development  iteration  should  closely  approximate  the  re¬ 
quired  form  factor  including  the  appropriate  levels  of  repair,  standardization, 
reliability,  maintainability,  safety,  human  engineering,  and  environmental  qualifica¬ 
tions.  The  development  model  produced  may  be  nothing  more  than  an  assembly  of 
existing  equipments  or  it  may  consist  of  extensively  developed  equipments. 

The  purposes  of  the  validation  phase  may  be  served  by  the  proof  of  feasibility 
of  a  key  concept,  device,  circuit,  or  equipment  in  combination  with  technical  analysis 
if  the  other  portions  of  the  envisioned  system  are  well  defined,  the  system  integration 
lias  a  low  risk  associated  with  it,  and  sufficient  engineering  data  are  available  for  fur¬ 
ther  development  and  any  contract  definition  requirements.  However,  tho  confidence 
achieved  through  advanced  development  can  greatly  reduce  the  technical  risk  associ¬ 
ated  with  follow-on  development.  Except  that  the  assembled  system  and  its  perform¬ 
ance  to  existing  requirements  are  at  issue  rather  than  the  feasibility  of  a  piece  of  the 
system,  the  considerations,  issues,  and  alternatives  in  advanced  development  are 
almost  identical  to  those  in  exploratory  development.  The  provision  of  documentation 
and  preservat:on  of  options  in  tho  succeeding  acquisition  phases  are  important.  To 
this  end,  compatibility  with  possible  procurement  alternatives  and  the  mqjor  issues  in 
the  transition  to  production  must  be  maintained.  Since  advanced  development  fre¬ 
quently  defines  values  for  parameters  in  the  technical  requirements,  it  is  important 
also  to  define  acceptable  tolerances  in  order  not  to  exclude  possible  existing  equip¬ 
ments. 


When  industry  development  is  pursued,  competition  should  be  maintained 
between  at  least  as  many  sources  as  will  be  solicited  in  the  follow-on  phase. 
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It  is  the  Government’s  responsibility  to  ensure  that  a  development  baseline  is 
clearly  defined  and  low-risk  for  follow-on  engineering  development. 

ENGINEERING  DEVELOPMENT 


Engineering  development  is  the  most  expensive  development  phase;  however, 
the  technical  risks  encountered  should  be  less  than  in  either  advanced  or  exploratory 
development.  Engineering  development  includes  the  product  design  and  production 
engineering  tasks  required  to  make  the  item  reproducible  while  maintaining  the  per¬ 
formance,  reliability,  maintainability,  environmental,  human  engineering,  etc.,  char¬ 
acteristics  which  were  determined  to  be  required  in  the  validation  phase.  The  techni¬ 
cal  risks  are  greatly  increased,  however,  when  an  advanced  development  phase  has 
not  been  completed  or  when  the  technical  requirements  parameters  have  not  been 
sufficiently  validated  to  have  established  acceptable  tolerances.  Tight  tolerances  are 
more  difficult  to  engineer  into  production  processes,  and  the  lack  of  solid  engineering 
data  increases  the  “fudge  factors”  an  engineer  utilizes  to  ensure  the  product  will  per¬ 
form  as  intended.  In  practice,  some  increased  risk  must  be  assumed  in  order  to  move 
ahead  in  the  acquisition  process.  Engineering  development  costs  are  higher  because 
of  (1)  the  scope  of  the  engineering  effort,  (2)  the  detailed  configuration  control  which 
must  be  maintained  on  production  processes  and  tooling  as  well  os  on  the  design  of 
the  equipment,  and  (3)  the  multitude  of  documentation  and  support  efforts  normally 
associated  with  this  phase— ILS  tasks,  technical  manuals,  provisioning  lists,  etc.  (see 
chapter  VI,  Transition  to  Production). 

The  selection  of  an  engineering  development  alternative  will  depend  on  the 
resolution  of  tho  mqjor  issues  in  the  transition  to  production  and  the  procurement 
mode  alternatives.  The  first  issue  will  be  in-house  versus  industiy  development;  the 
factors  in  this  decision  include  the  extent  of  industry  involvement  in  prior  phases,  the 
appropriateness  of  in-house  involvement,  the  capabilities  of  available  in-house 
resources  versus  industry  sources,  the  government  needs  for  precise  control  of  the 
technical  effort  in  conjunction  with  other  program  factors,  and  the  percent  of  the  sys¬ 
tem  requiring  development.  If  industry  has  played  only  a  support  rolo  in  prior  phases, 
in-house  involvement  is  appropriate  (necessary),  and  capable  in-house  resources  are 
available,  tho  in-house  execution  should  be  selected.  Tho  necessity  of  in-houso 
involvement  may  arise  from  tho  factors  under  which  in-house  development  is  appro¬ 
priate  (see  chapter  VI),  from  government  need  for  control,  or  from  the  primacy  of  the 
system  integration  responsibilities  when  only  a  small  portion  of  the  system  requires 
development.  Normally,  an  industry  alternative  will  be  selected;  this  will  require 
resolution  of  the  mqjor  issues  prior  to  the  development  contract  solicitation.  In  addi¬ 
tion,  the  following  issues  must  be  considered:  (1)  requirements  for  competition,  (2) 
program  cost  constraints,  and  (3)  prior  industry  involvement. 

If  industry  has  been  involved  in  prior  phases,  will  those  contractors  be  capable 
of  meeting  the  remaining  program  requirements  for  production  and  support,  and  will 
they  be  adequate  in  number  to  support  any  needed  competition  dictated  by  the  pro¬ 
curement  mode?  A  competitive  base  must  be  maintained  up  to  tho  point  that  all  fol- 
low-on  costs  for  procurement,  reprocurement,  and  contractor  support  can  be  fixed 
within  contractually  binding  limits.  If  new  contractors  are  to  be  brought  in,  provi¬ 
sions  must  be  made  to  educate/indoctrinate  the  contractors  to  the  program  require- 
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ments,  key  program  elements,  acceptable  tradeoffs  which  may  not  have  been 
specified,  etc.  If  a  sufficient  number  of  contractors  have  participated  in  advanced 
development,  competitive  selection  may  be  considered  for  engineering  development.  If 
an  insufficient  number  of  contractors  are  available  to  support  the  procurement  mode 
needs,  provisions  should  be  made  to  execute  the  preproduction  phase  in-house,  or  at 
least  to  make  the  successful  validation  of  the  design  data  package  part  of  the  accep¬ 
tance  procedure  for  the  engineering  development  contract  in  a  leader-follower  pro¬ 
gram. 

Further  requirements  for  competition  may  arise  from  the  risks  associated  with 
the  technical  approaches  identified  in  the  prior  phases.  If  the  technical  risk  is  low  — 
i.e.,  the  mqjor  system  equipments  have  been  previously  developed  and  used  together 
on  a  prior  program  and  require  only  adaptational  development  for  this  program  —  a 
single  technical  appros  :h  may  be  pursued.  If  a  moderate  risk  exists  —  eg.,  the  mqjor 
system  equipments  have  been  previously  developed  but  never  before  integrated  in  this 
way  —  a  primary  technical  approach  should  suffice,  but  alternative  approaches 
should  be  identified  which  can  provide  an  acceptable  product  with  minimal  transi¬ 
tional  costs  should  the  preferred  approach  bog  down.  If  a  higb  risk  exists  —  e.g.,  one 
or  more  mqjor  system  equipments  requires  development  —  parallel  technical 
approaches  should  be  pursued. 

Program  cost  constraints  can  be  mqjor  determining  factors,  especially  for  low 
priority  programs.  Funds  may  not  be  available  to  support  competitive  development  or 
an  adequate  preproduction  (design  validation)  phase,  and  conditions  may  not  lend 
themselves  to  functional  specification  procurement.  Assuming  in-house  development 
cannot  be  followed  even  though  it  may  be  the  best  alternative  under  these  conditions, 
and  assuming  no  additional  funds  can  be  identified  for  design  validation,  the  procure¬ 
ment  must  be  converted  to  a  combined  design-to-cost  and  life-cycle-cost  procurement. 
Design-to-cost  procurements  attempt  to  control  acquisition  costs  and  contractor- 
supplied  support  costs;  lifecycle-cost  procurements  attempt  to  control  operations  and 
support  costs.  Both  types  of  procurement  are  discussed  below.  This  type  of  combined 
procurement  is  risky;  the  contractor  must  be  contractually  obligated  to  absorb  loss 
due  to  bis  failure.  If  such  a  contract  is  not  feasible  (because  of  the  extent  of  develop¬ 
ment  uncertainty),  and  if  contractor  default  cannot  be  allowed  for  any  reason,  the 
government  is  better  off  canceling  the  acquisition  altogether. 

DESIGN-TO-COST  (DTC)  PROCUREMENT 

Program  cost  constraints  in  the  procurement  pliase  give  rise  to  dcsign-to-cost 
(DTC)  procurement  modes.  In  DTC,  acquisition  costs  ore  fixed  prior  to  engineering 
development  and  provisions  for  proving  the  cost  targets  have  been  achieved  and  are 
incorporated  into  the  acceptance  procedures;  contractual  incentives  are  often 
employed.  Very  wide  and  notably  achievable  performance  tolerances  are  normally 
required  because  cost  is  the  prime  design  factor  and  performance  is  traded  off  to 
achieve  that  cost  within  acceptable  limits.  The  advanced  development  phase  is  usu¬ 
ally  required  to  identify  reasonable-cost  targets  and  acceptable  performance  parame¬ 
ters.  Procurement  quantities  to  support  multiyear,  multisource  buys  are  required. 

The  Joini  Design-to-Cost  Guide  (AMCP  700-6/NAVMAT  P5242/AFLCP- 
AFSCP  800-19)  contains  guidelines  in  the  formulation  and  execution  of  a  DTC  pro¬ 
curement. 
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LIFE-CYCLE-COST  (LCC)  PROCUREMENT 


Life-cycle-cost  (LCC)  procurements  are  useful  when  the  equipment  service  life 
is  long  enough  that  support  costs  become  significant;  generally,  only  a  few  years  is 
required,  as  these  costs  can  accumulate  rapidly.  A  reasonably  accurate  LCC  model  for 
the  system  is  required,  and  the  award  of  the  contract  is  made  to  the  lowest  bid  which 
results  in  the  least  overall  costs  to  the  government.  A  number  of  procurement  modes 
are  available  along  this  theme.  The  Life-Cycle  Costing  Procurement  Guide  (DoD 
LCC-1)  is  useful  in  constructing  the  model,  and  the  Life-Cycle  Costing  Guide  for  Sys¬ 
tem  Acquisitions  (DoD  LCC-3)  discusses  the  procedural  aspects,  including  some  sug¬ 
gested  procurement  modes.  LCC  procurement  can  be  particularly  effective  when 
long-term  warranty  provisions  are  integrated  into  the  procurement.  LCC  procure¬ 
ment  effectiveness  is  greatly  enhanced  when  large  reprocurements  can  be  made  over 
many  years  to  multiple  sources. 
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XVIII.  INSTALLATION  PLANNING 


This  section  provides  information  to  assist  the  project  manager  in  plan¬ 
ning  for  system  and  equipment  installations  on  Navy  platforms  -  tem¬ 
porary,  for  tests  and  evaluations;  and  permanent,  for  service  use. 


TEMPORARY  INSTALLATIONS 


The  primary  purpose  of  temporary  installations  (under  1  year)  is  to  subject 
equipments  to  the  operational  environment  for  a  realistic  test  and  evaluation  (see 
chapter  XIX).  However,  short-term  operational  requirements  may  also  dictate  the 
temporary  installation  of  equipment  in  service  platforms,  and  occasionally  it  will  be 
necessary  to  install  instruments  to  measure  operational  environments.  Whatever  the 
circumstance,  the  proper  management  of  a  temporary  installation  is  important  to  the 
success  of  the  project  and  to  the  operation  of  targeted  platform. 

Tho  project  is  responsible  for  supplying  all  the  resources  necessary  to  accom¬ 
plish  and  to  disestablish  a  temporary  installation.  The  funding  is  in  excess  of  any  spe¬ 
cial  operating  and  observer  costs  for  operational  tests  and  evaluation.  The 
installation  personnel  are  under  project  tasking.  It  is  incumbent  on  the  project  man¬ 
ager  to  plan  for  these  requirements  well  ahead  to  ensure  that  the  budget  is  adequate 
to  support  the  installation  tasks. 

The  most  difficult  task  in  the  temporary  installation  is  obtaining  Fleet  serv¬ 
ices  and  coordinating  them  with  the  availability  of  tho  equipment  to  be  installed. 
Fleet  services  are  requested  through  CNO  in  accordance  with  OPNAVINST  39G0.10. 
(However,  minor  services  are  sometimes  available  informally  through  special  labora¬ 
tory-type  commander  relationships  such  as  the  Navy  Science  Assistance  Program 
(NSAP)  and  other  Floot  assistance  functions.)  These  formal  procedures  are  designed 
to  coordinate  the  burden  on  Fleet  operations  and  should  be  followed  for  all  formal 
test  programs. 

Tho  other  tasks  are  as  follows: 

1.  Preliminary  planning  —  establishes  the  scope  of  the  installation;  the  num¬ 
ber  of  equipments  involved;  the  interfaces  to  platform  systems  such  as 
power,  air  conditioning,  navigation  systems,  and  sensors;  the  types  of 
materials  needed;  tho  skills  needed  to  execute  the  installation;  tire  types  of 
support  required  for  the  equipment  while  it  is  installed;  and  the  travel  and 
transportation  requirements  for  the  installation  personnel  and  the  equip¬ 
ment. 

2.  Installation  survey  —  accomplished  well  ahead  of  the  planned  installation 
date,  the  survey  identifies  precise  equipment  installation  sites,  mounting 
requirements,  relationships  with  platform  interfaces,  cable  routings  and 
distances,  and  man-hour  estimates  for  the  different  installation  skills. 

3.  Detailed  planning  —  tasking  of  tho  installation  team  (project  personnel, 
contractor  field  representatives,  shipyard,  etc.),  procurement  of  required 
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materials,  documenting  the  installation  procedures,  establishing  checkout 
procedures,  and  determining  ripout  procedures. 

(Plans  for  temporary  submarine  installations  must  be  submitted  to  the 
cognizant  NAVSEA  Ship  Logistics  Manager  (SLM)  for  approval.) 

4.  Installation  and  checkout 

5.  Ripout  —  it  was,  after  all  a  temporary  installation. 

It  is  wise  to  use  the  installation  survey  to  establish  good  relations  with  platform  per¬ 
sonnel  and  to  keep  them  informed  of  your  intentions.  They  can  be  a  valuable  source 
of  information  regarding  unsuspected  problems  or  unusual  platform  characteristics 
and  can  also  provide  limited  assistance  in  the  installation,  operation,  and  mainte¬ 
nance  of  the  equipment  beyond  the  well-defined  scope  of  operational  testing. 

No  matter  which  type  of  platform  —  ship,  aircraft,  vehicle,  or  whatever  —  is 
involved,  the  thorough  advanced  planning  and  detailed  execution  of  a  temporary 
installation  are  essential.  The  failure  to  plan  adequately  will  result  in  badly  slipped 
schedules  or  poor  equipment  operation.  The  installation  plan  should  include  steps  to 
ensure  that  the  installed  equipment  does  not  interfere  with  (or,  conversely,  is  not  sus¬ 
ceptible  to  interference  by)  equipments  already  on  the  platform,  many  problems  can 
be  created  by  mounting  equipments  too  close  or  overloading  interfacing  systems  and 
the  like  which  a  sensible  approach  can  avoid.  Also,  the  installation  plan  should  show 
contingency  plans  covering  changes  in  schedule,  in  either  equipment  or  platform 
availability.  As  with  any  other  project  task,  plan  ahead,  anticipate  and  minimize  risks 
and  execute  the  plan  to  maintain  the  pattern  of  success. 


PERMANENT  INSTALLATIONS 


Navy  operational  requirements  tend  to  grow  ever  more  complex.  Now  systems 
and  equipments  and  improvements  to  existing  equipments  are  constantly  demanded. 
Thus,  Navy  platforms  are  constantly  undergoing  change.  Without  change  control, 
installation  of  new  equipment  and  alternation  of  old  would  eventually  become  impos¬ 
sible.  Table  XVIII-1  summarizes  the  types  of  alterations  and  changes  affecting  Navy 
platforms. 


Table  XVIII- 1.  Alterations  and  changes. 


Type  of 
Change 

Cognizant 

SYSCOM 

Initiating 

Document 

Funding 

Source 

Reference 

SHIPALT 

Military 

Improvement 

Sponsoring 

SYSCOM 

PMI 

FMP 

OPNAVINST 

4720.2D, 

NAVSEAINST 

4720.3 

Technical 

Improvement 

ECP 

FMP 

MIL-STD-480, 

NAVSEAINST 

4720.3 
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Table  XVIH-1.  Alterations  and  changes  (continued). 


Type  of 
Change 

Cognizant 

SYSCOM 

Initiating 

Document 

Funding 

Source 

Reference 

QRC  SHIPALT 

PMI  or  ECP 
as  appro¬ 
priate  and 

Ltr  of  Justi¬ 
fication 

ORDALT 

NAVSEA 

ECP 

project 
(until  nor¬ 
mal  budget¬ 
ing  occurs) 
FMP 

(see  SHIPALTS) 

SPALT 

NAVSEA08 

ECP 

OMN/FMP 

SSPIP4720.1C 

A1RALT 

NAVAIR 

ECP 

OMN/PAMN 

(NAVAIRINST 

4720.  IB) 

for  modifications 

SPECOMALT 

NAVELFX 

ECP 

OMN 

NAVELEXINST 

4720.1, 

NAVELEXINST 
11000.  IB 

Shore  site 
alteration 
(BESEP) 

NAVELEX 

ECP 

OMN 

NAVELEXINST 
11000.  IB 

Electronic 
field  change 

NAVELEX 

ECP 

(soe  MIL-F- 
17G55) 

M1L-F-17G55C 

SHIP  ALTERATIONS  (SHIPALTS) 


All  proposed  alterations  classified  a?  military  improvements  and  technical 
improvements  to  the  ship,  to  hull  equipments,  or  affecting  space  layout  or  configura¬ 
tion  are  handled  by  SHIPALT.  Alterations  to  ordnance  equipment  (ORDALTs)  are 
handled  as  SHIPALTs. 

Ship  alterations  are  controlled  tlirough  the  Fleet  Modernization  Program 
(FMP).  The  Naval  Sea  Systems  Command  (NAVSEA)  is  the  designated  executive 
agent  for  the  FMP.  For  alterations  of  minor  impact,  CNO  delegates  responsibility  for 
configuration  control  to  the  cognizant  systems  command. 
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These  programs  are  structured  so  that  they  may  be  properly  planned,  docu¬ 
mented,  scheduled,  funded,  and  executed.  Review  of  engineering  effort  and  cost- 
performance  tradeoff  are  built  in.  The  project  manager  is  well  advised  to  allow  ade¬ 
quate  time  for  all  these  essential  steps  in  what  by  its  nature  must  be  a  lengthy ,  nd 
serial  procedure. 

FLEET  MODERNIZATION  PROGRAM 


Ship  alteration  programs  requiring  depot-level  capability  or  budget  action  for 
the  procurement  of  special  material  are  scheduled  through  the  Fleet  Modernization 
Program  (FMP).  The  FMP  is  promulgated  by  CNO  and  managed  by  NAVSEA  (see  fig. 
XVIII- 1).  It  is  implemented  via  the  semiannual  FLTMOD  conference.  Inputs  to  the 
conference  are  consolidated  in  the  Military  Improvement  Plan  (MIP),  the  Technical 
Improvement  Plan  (TIP),  and  the  Amalgamated  MIP/TIP  (AMT)  —  one  AMT  for  each 
ship  class. 


4 - 2  YEARS 


SHIPALT 

OROALT 


1  YEAR 


Figure  XVIII-1.  Fleet  modernization  program  (FMP). 


The  MIPs  and  TIPs  are  working  papers  used  by  the  SYSCOMc  and  type  com¬ 
manders  in  preparing  for  FLTMOD  conferences  and  by  the  Ship  Acquisition  and 
Improvement  Panel  (SAIP)  as  inputs  to  the  AMTs. 

The  AMTs  are  developed  by  the  SAIP-Working  Group,  with  the  technical  sup¬ 
port  of  the  SYSCOMs.  They  are  based  on  consideration  of  ship  and  personnel  safety, 
new  systems  and  programs,  fleet  capability,  and  CNO  policy.  They  consolidate  and 
prioritize  MIP  and  TIP  items  and  make  it  possible  to  establish  realistic  plans  for 
their  accomplishment.  They  are  used  to  determine  alterations  to  be  added  when 
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additional  funds  become  available  and  alterations  to  be  eliminated  when  programs 
are  reduced.  They  constitute  the  basic  FMP  documentation. 

SUBMITTING  PROPOSALS  FOR  ALTERATIONS 


The  type  of  initiating  document  which  must  be  submitted  for  a  proposed 
alteration  depends  upon  the  nature  of  the  alteration  (see  table  XVIII-1).  The  Pro¬ 
posed  Military  Improvement  (PMI)  i '  the  initiating  document  for  a  SHIPALT  which 
effects  a  military  improvement  (one  of  nature  or  magnitude  such  as  to  increase  its 
operational  capabilities).  The  Engineering  Change  Proposal  (ECP)  is  the  initiating 
document  for  a  SHIPALT  which  effects  a  technical  improvement  (one  which  concerns 
the  safety  of  personnel  or  the  reliability,  maintainability,  and  efficiency  of  installed 
equipment).  Ordnance  alternations  (ORDALTs)  and  Special  Project  Alterations 
(SPALTs)  (nuclear  propulsion  plant  and  tender  nuclear  support  facility  alterations) 
are  also  initiated  by  ECP. 

It  is  important  that  the  project  manager  fully  identify  critical  installation 
parameters  and  procedures  upon  requesting  an  alteration.  The  estimated  increase  in 
weight  and  vertical  moment  should  be  stated,  together  with  a  recommendation  for 
the  removal  of  sufficient  weight  to  compensate  for  it.  If  an  alteration  involves  a 
reduction  in  space  or  living  facilities  normally  available,  the  amount  of  the  reduction 
and  the  reason  for  accepting  it  should  be  stated.  After  a  PMI  is  approved,  and  upon 
request  of  the  appropriate  SYSCOM,  the  PMI  originator  develops  information  to 
serve  as  guidance  for  the  SYSCOM  in  matters  concerning  plans,  support  require¬ 
ments,  equipment/material,  test  equipment,  test  and  checkout,  weight  and  moment, 
training,  manpower,  and  personnel. 

In  the  interests  of  standardization,  approved  alterations  are  normally  author¬ 
ized  for  all  ships  to  which  they  are  applicable. 

APPROVAL  OF  ALTERATIONS 


Military  Improvements 

The  decision  to  incorporate  a  Proposed  Military  Improvement  (PMI)  rests 
with  CNO  (SAIP).  The  SAIP  —  Working  Group  reviews  and  determines  acceptance  at 
the  FLTMOD  conference.  Due  regard  is  given  to  ship  situation  resulting  from  altera¬ 
tions  already  included  in  tho  AMT  for  the  particular  ship  class. 

Approved  PMIs  aro  entered  officially  in  the  MIP  and  are  considered  for  appro¬ 
priate  priority.  If  doubt  exists  as  to  whether  an  improvement  is  military  or  technical, 
the  matter  is  referred  to  CNO  (SAIP)  for  resolution.  PMIs  not  approved  are  returned 
to  the  originator. 

Technical  Improvements 

Requests  for  approval  of  proposed  technical  improvements  are  forwarded  to 
tho  cognizant  SYSCOMs  as  ECPs.  The  SYSCOM  reviews  the  alteration  under  the 
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command  of  the  SYSCOM  Change  Control  Board  (CCB).  The  alteration  is  reviewed 
for  concurrence  in  classification.  The  CCB  approves  or  disapproves,  assigns  priorities, 
and  determines  whether  the  improvement  will  be  handled  as  a  SHIPALT.  If  it  is  to  be 
handled  as  a  SHIPALT,  the  approved  ECP  is  forwarded  to  NAVSEA,  where  it  is 
officially  entered  in  the  Technical  Improvement  Plan  (TIP)  upon  acceptance  and  con¬ 
sidered  for  priority  at  the  next  FLTMOD  conference.  The  TIP  is  approved  by  the  cog¬ 
nizant  SYSCOM. 

Ordnance  Alterations  (ORDALTs)  are  handled  in  much  the  same  manner,  by 
the  cognizant  ordnance  section  of  NAVSEASYSCOM. 

If  the  technical  improvement  is  an  Alteration-Equivalent-to-Repair  (AER) 
(substitution  of  different  but  standard  materials  or  parts  of  later  design;  strengthen¬ 
ing  for  greater  reliability;  or  other  minor  modification  or  replacement),  it  may  be 
approved  and  authorized  for  accomplishment  by  the  fleet  commander  to  the  extent 
that  such  authority  has  been  delegated  by  the  SYSCOM.  A  SHIPALT  is  not  neces¬ 
sary. 

When  a  technical  improvement  is  disapproved,  the  originator  is  notified. 
Execution  of  the  FMP 

The  FLTMOD  conference  takes  place  each  year  in  May  and  November.  The 
FMP  is  published  in  July  and  January.  It  contains  firm,  proposed,  revised,  and  tenta¬ 
tive  listings.  It  constitutes  the  CNO-approved  program  for  modification  and  improve¬ 
ment  of  the  ships  of  the  Fleet. 

Fleet  and  typo  commanders,  OPNAV,  and  NAVSEASYSCOM  and  other  com¬ 
mands  participate  in  the  formulation  and  review  of  this  program.  NAVSEA  collabo¬ 
rates  closely  with  all  concerned  to  achieve  efficient  execution  of  changes  in 
requirements,  available  funding,  material  delivery  schedules,  overhaul  schedules,  and 
the  other  factors  which  bear  upon  stability.  NAVSEA  issues  a  Material  Supplement 
for  each  semiannual  FMP  covering  the  FMP  Exocution  and  Budget  Year  and  author¬ 
izes  advance  planning  by  Planning  and  Engineering  for  Repairs  and  Alterations 
(PERA)  and  overhaul  yards  as  required. 

At  the  time  of  approval  and  promulgation  of  the  FMP  by  OPNAV,  distributin' 
of  the  FMP  is  made  under  the  cover  of  CNO  letter  of  promulgation.  This  letter 
advises  the  fleets  regarding  actions  to  be  taken  if  program  a<(justmcnts  become  neces¬ 
sary  to  meet  changing  operation  conditions. 

Requests  for  changes  to  the  approved  FMP  from  Forces  Afloat,  Program  Man¬ 
agers,  or  other  sources  must  be  directed  to  CNO,  with  compensation  and  justifica¬ 
tion,  via  the  typo  and  fleet  commander  where  applicable,  with  information  copies  to 
NAVSEA. 

QRC/RDC  SHIPALTS 

Requirements  which  are  identified  out  of  phase  with  the  normal  programming 
and  budgeting  cycle  and  which  are  considered  by  the  SAIP  —  Working  Group  to  be  of 
higher  priority  than  existing  requirements  are  accommodated  on  a  case  basis.  Quick 
Reaction  Capability  SHIPALTs  are  initiated  by  letter  of  justification.  Sponsors  of 
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these  Quick  Reaction  Capability  and  Rapid  Development  Capability  requirements 
must  assure  that  normal  programming  and  budgeting  are  commenced  immediately. 

SHIP  LOGISTICS  DIVISION  APPROVAL 

All  now  shipboard  installations  and  configuration  changes  that  affect  the  in¬ 
terface  between  a  ship  and  its  systems  and  equipments  are  reviewed  by  the  cognizant 
NAVSEA  Logistics  Division  (SLD).  This  includes  all  long-term  (over  1  year)  tempo¬ 
rary  installations  such  as  mission-oriented  installations,  OP/Tech  Installations,  and 
R&D  installations. 

The  installation  plan  should  be  submitted  through  the  cognizant  type  com¬ 
mander  well  in  advance  of  the  required  approval  date  in  order  to  allow  adequate  time 
for  NAVSEA  review.  It  should  include  the  following  information:  project  title,  secu¬ 
rity  classification,  sponsoring  activity,  applicable  ship  and  type  commander,  installa¬ 
tion  du"-'  jn,  installation  site,  installation  activity,  installation  technical  support 
requirements,  installation  impact  data,  and  installation  drawings  (see  appendix  A). 

NAVSEA  reviews  the  installation  plan  for  technical  adequacy,  technical  accu¬ 
racy,  and  impact  on  ship  safety,  personnel  safety,  military  capability,  system  inter¬ 
face,  equipment  arrangement,  stability,  and  ship  systems,  and  sends  a  message  or 
speed  letter  to  cognizant  activities  on  approval. 

Aircraft  Alterations  (AIRALTS) 

Aircraft  alterations  and  changes  are  managed  by  the  Naval  Air  Systems  Com¬ 
mand  (NAVAIR).  Proposed  alterations  are  submitted  via  ECP  to  the  cognizant 
airframe  type  desk  in  NAVAIR.  The  NAVAIR  CCB  acts  on  each  ECP  and  the  origina¬ 
tor  is  notifir  J.  Normally,  approved  alterations  are  accumulated  and  incorporated  en 
masse  into  tne  airframe  as  a  change  of  model  (o.g,,  F-4B  becomes  F-4J).  Mass  changes 
are  coordinated  with  new  aircraft  procurements  and  aircraft  overhaul  schedules. 
Changes  which  can  be  accomplished  by  direct  black  box  substitution  are  programmed 
to  coincide  with  overhauls.  Alterations  which  are  safety  critical  are  scheduled  on  a 
priority  basis. 

Shore  Site  Alterations 

Shore  site  alterations  include  Base  Electronic  System  Engineering  Plan 
(BESEP)  and  Special  Communications  System  Alterations  (SPECOMALTS).  Both  are 
administered  by  the  Naval  Electronic  Systems  Command  (NAVELEX).  Proposed 
changes  are  submitted  via  ECP  and  acted  upon  by  the  NAVELEX  CCB.  BESEPs  are 
specifically  managed  by  the  NAVELEX-assigned  Field  Technical  Authority  (FTA). 
NAVELEX  PME-117  manages  SPECOMALTs.  (All  non-SPECOM  alteration  propos¬ 
als  should  be  submitted  to  NAVELEX  headquarters,  SPECOM  alteration  proposals 
should  be  directed  to  PME-117.) 

FIELD  CHANGES 

Field  changes  are  developed  for  the  purpose  of  improving  equipment  after 
delivery  to  the  government  with  respect  to  performance,  operational  characteristics, 
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or  maintenance  features.  The  change  may  be  a  simple  wiring  or  mechanical  modifica¬ 
tion  to  an  existing  equipment  and  consist  entirely  of  instructions  for  making  the 
change,  or  it  may  be  more  extensive,  requiring  circuit  changes  and  the  removal  and/ 
or  substitution  of  parts.  Material  supplied  or  required  is  listed  in  the  field  change 
bulletin,  which  also  provides  step-by-step  instructions  for  accomplishing  the  change. 

Field  changes  bear  designations  of  type,  class,  and  priority  which  are  defined 
in  MIL-F-17655C.  Type  indicates  extent  to  which  parts  are  supplied;  class  indicates 
funding  and  installation  responsibility;  and  priority  indicates  urgency  of  accomplish¬ 
ment. 


Changes  are  initiated  by  the  submission  of  an  Engineering  Change  Proposal 
(ECP)  (see  MIL-STD-480  or  -481).  The  originator  may  be  any  agency  involved  in  the 
design,  production,  maintenance,  or  operation  of  a  system  or  subsystem.  If  the  pro¬ 
posed  change  requires  a  related  change  to  equipment  or  material  under  the  cogni¬ 
zance  of  another  command,  a  Request  for  Conjunctive  Alteration  is  submitted  with 
the  ECP.  The  originator  must  ensure  that  appropriate  drawings,  sketches,  and  refer¬ 
ence  data  are  submitted  to  the  Cognizant  Technical  Division,  the  Coordinating  Activ¬ 
ity,  the  Change  Control  Board  (CCB),  and  other  reviewing  activities.  The  originator 
must  also  assure  that  copies  of  the  ECP  are  distributed  simultaneously  to  all  review¬ 
ing  authorities. 

The  SYSCOM  CCB  reviews  and  evaluates  proposed  changes  and  recommends 
approval  or  disapproval  to  the  cognizant  SYSCOM  Project  Office.  The  Project  Office 
makes  the  final  determination. 

If  an  ECP  is  disapproved,  all  affected  activities  are  notified. 

ECP  Approval 

The  systems  command  project  offices  and  CCBs  base  their  approval  upon  the 
following  factors: 

1.  Need  for  the  change,  adequacy  of  justification 

2.  Cost-effectiveness 

3.  Funding  availability 

4.  Effects  on  training  installations 

5.  Impact  on  logistic  support 

6.  Proposed  installation  schedule 

7.  Adequacy  of  the  design  and  procurement  documentation 

The  approval  or  disapproval  will  be  documented  by  the  CCB  and  forwarded  to  the 
originator. 

Other  Notes  on  Installations 

Installation  plans  contribute  to  the  equipment  technical  manuals.  Refer  to 
MIL-M-15071  for  specific  instructions. 
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In  formulating  installation  plans,  remember  safety  and  maintenance.  The 
installation  should  not  create  hazards  for  the  operators  and  maintainors,  and  it 
should  provide  for  ample  maintenance  access. 
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APPENDIX  A:  INSTALLATION  PLAN 


The  objective  of  the  installation  plan  is  to  provide  all  the  necessary  informa¬ 
tion  to  perform  a  cost  and  feasibility  (C&F)  study.  This  ensures  that  all  elements 
impacting  the  ship  and  its  various  systems  are  considered  and  identified.  This  infor¬ 
mation  is  made  available  to  those  concerned  for  decision-making  purposes  during  the 
process  leading  to  an  accomplished  ship  improvement.  It  must  be  recognized  that  the 
validity  of  a  C&F  study  depends  upon  the  accuracy  of  the  technical  input;  therefore, 
the  originator  is  responsible  for  providing  technical  data  of  the  highest  possible  valid¬ 
ity. 

Valid  information  considering  each  of  the  following  items  is  desired  with  each 
proposed  improvement. 

I.  PLANS 

A.  Block  Diagrams.  Show  the  function  of  each  basic  unit  with  relation  to 
other  units  comprising  the  system.  Indicate  normal  inputs  and  outputs  of 
each  unit  in  tho  system.  Include  block  diagrams  of  all  units  to  be  furnished 
plus  all  other  units  required  to  complete  the  installation,  including  rocks, 
junction  boxes,  motor  generators,  and  antennas. 

B.  Arrangement.  Include  arrangement  for  each  electronic  space  and  for  all 
other  spaces  in  which  an  appreciable  amount  of  electronic  equipment  is 
installed.  Prepare  a  plan  view,  an  elevation  view,  and  other  views  as  neces¬ 
sary  to  show  the  arrangement  clearly.  Structural  members,  ventilation 
ducts,  piping,  and  equipment  other  than  electronic  equipment  shall  be 
indicated.  Also  show  large  transmission  lines.  Give  details  of  equipment 
characteristics  that  influence  the  relative  locations  of  units  to  optimize 
operation,  maintenance,  reliability,  and  safety. 

C.  Stowage.  Give  typical  stowage  requirements,  physical  dimensions,  and  en¬ 
vironmental  requirements  for  major  spare  parts.  Include  stowage  area  for 
test  equipment,  special  tools,  technical  manuals,  etc. 

D.  Ripout 

1.  Removal  instructions.  Include  all  equipment,  cables,  piping,  and  foun¬ 
dations  to  be  removed.  The  following  should  be  provided: 

a.  Location  of  item 

b.  Name  of  item 

c.  Disposition 

d.  Cable  run  and  length 

e.  Weight  of  item  (note  change  of  weight  and  moment) 

2.  Compartment  and  access  modification.  Include  all  structural  members, 
bulkheads,  etc.,  to  be  removed.  All  drawings  should  resemble  arrange¬ 
ment  drawings. 
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E.  Interfaces.  All  interconnection  requirements  with  other  sets,  power 
sources,  etc.,  shall  be  shown  and  identified.  Include  external  cables,  rf 
transmission  lines,  piping  for  water  cooling  and  heating,  piping  for  air  and 
other  services,  and  any  types  of  interconnections  among  units  and  between 
units  and  power  sources.  Identify  necessary  concurrent  or  prior  completion 
items  such  as  SHIPALTs,  Field  Changes,  and  Mods. 

II.  SUPPORT  REQUIREMENTS 

A.  Power.  Indicate  type  of  power  to  be  used.  For  electrical  power  give  the 
requirements  for  the  complete  equipment  indicating  for  each  power  input: 
power  typo  I,  II,  or  III  (as  defined  by  MILrSTD-761),  voltage,  phase,  fre¬ 
quency,  current,  power  factor,  and  power.  Power  for  starting,  operating, 
standby,  and  secured  conditions,  as  applicable,  shall  be  specified.  Allowable 
variations  in  voltage  and  frequency  shall  also  be  specified. 

B.  Cooling.  Indicate  type  of  cooling  or  heating,  whether  water  or  air.  Give  the 
amount  of  heat  to  be  dissipated.  A  diagram  should  depict  the  routing, 
direction  of  flow,  components  with  physical  location,  arrangement,  and 
other  characteristics  or  constraints.  All  units  to  be  cooled  or  heated  shall 
be  shown. 

C.  Air  conditioning.  Give  the  ambient  temperature  range  and  limits  of  rela¬ 
tive  humidity  for  which  the  equipment  is  designed.  Indicate  the  location, 
size,  and  arrangement  of  ventilation  openings  and  external  ventilation 
ducts  supplied  or  required,  indicating  capacity  and  direction  flow.  Inter¬ 
face  to  ship  services  shall  bo  clearly  specified  and  shall  include  duct  size, 
material,  allowable  particle  size,  allowable  moisture  environment,  allow¬ 
able  velocity,  filter  characteristics,  and  pressure  required. 

D.  Special  Material.  Make  a  list  of  special  material  such  as  cables,  special 
tools,  waveguides,  mounting  equipment  and  hardware,  connectors,  bulk 
material,  parts,  loose  hardware,  test  equipment,  and  other  support  items 
needed  to  complete  the  installation  which  are  not  part  of  the  installation 
package. 

E.  Shock.  Indicate  any  special  part  arrangements,  mounting  techniques,  or 
joining  methods  to  harden  equipment  against  shock  and  vibration.  Give 
clearances  required  for  shock  and  vibration  excursions  on  shock-mounted 
equipment. 

F.  Safety.  Establish  safety  requirements  to  promote  maximum  safety  for  per¬ 
sonnel  and  equipment  during  installation. 

1.  Identify  potential  hazards  and  methods  planned  to  eliminate  or  control 
them. 

2.  Outline  undefined  areas  requiring  guidance  or  decisions. 

3.  Describe  technical  risks  or  problems  in  design. 

G.  Stowage.  Prepare  a  list  of  all  installation  material  such  as  spares,  techni¬ 
cal  manuals,  and  tools  to  be  stored  in  stowage  areas. 
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H.  Planned  Maintenance  System  (PMS).  Express  the  planned  maintenance 
requirements  of  the  system,  subsystem,  or  component  (see  OPNAV 
43P2  and  OPNAVINST  4790.4).  The  planned  maintenance  require¬ 
ments  should  include  the  following:  cleaning,  inspection,  lubrication, 
replacement,  functional  test,  adjustment  and  alignment,  calibration, 
and  systems  check  (see  MIL-P-28759). 

III.  EQUIPMENT/MATERIAL 

A.  Description.  Give  the  weights  and  space  requirements  of  the  new  and 
replaced  (if  any)  installation.  Also  mention  any  constraints  that  may  exist. 

B.  Special  Instructions.  Mention  all  special  instructions  such  as  handling 
(location  of  handling  attachments,  crating),  mounting  (shock  mounting, 
special  clearances,  mounting  surface  and  hardware),  cable  entrance  sway 
bracing,  etc. 

C.  Standard  Stock  Items  versus  New  (Contract)  Items.  Consider  cost  tradeoffs 
in  logistic  requirements  for  supporting  any  new  material  or  equipment 
involved  and  retrofitting  the  new  material  equipment  to  existing  systems 
already  deployed. 

D.  Indication  of  Spare  Support  for  New  Equipment.  List  the  sources  of  all 
spare  support  materials  and  equipment  whethor  it  bo  the  government  Fed¬ 
eral  Stock  System  or  a  contractor. 

E.  Availability.  Make  a  brief  statement  of  the  availability  status  of  the  equip¬ 
ment  or  material.  Include  when  available  for  installation,  confidence  of 
this  date,  status  of  tech/op  ovals  or  service  approval,  procurement  lead 
times,  and  any  constraints  that  could  modify  this  status. 

F.  Technical  Manuals.  Technical  Manuals  shall  be  available  upon  installation 
date  and  provide  information  such  as  instructions  for  installation,  opera¬ 
tion,  and  maintenance. 

IV.  TEST  EQUIPMENT 

A.  General  Purpose.  Prepare  a  list  of  all  general-purpose  test  equipment  to  be 
used.  Include  the  official  name  or  nomenclature,  identifying  number,  and  a 
brief  description  of  the  use  of  the  item. 

B.  Special.  Prepare  a  list  of  all  special  tools  and  special  test  equipment  to  be 
used.  Special  tools  and  test  equipment  are  defined  as  those  not  listed  in 
the  Federal  Supply  Catalcg.  An  illustration  and  description  of  special 
items  required  shall  be  provided  for  identification. 

V.  TEST  AND  CHECKOUT 

Prepare  installation  and  checkout  test  plans  and  procedures  to  ensure  that  all 
equipment  is  physically  and  functionally  checked  out  and  to  demonstrate  the  ade¬ 
quacy  of  each  facility  installation.  Information  will  be  obtained  from  such  sources  as 
specifications,  related  test  documentation  used  in  the  development  test  program,  and 
relevant  technical  documentation  from  any  agency  concerned  with  the  installation 
testing. 


XVHI-13 


The  test  plan  section  shall  describe  the  overall  installation  test  program  and 
should  include: 

1.  Description  of  the  equipment  and  facility  configuration  at  each  site 

2.  Scope  of  testing 

3.  Objectives  of  each  test 

4.  Roles  and  of  all  mqjor  participants  including  the  government 

5.  Support  requirements  including  test  instrumentation  and  facilities 

6.  Milestones  and  scheduling 

7.  Security  guidelines 

Detailed  test  procedures  shall  be  provided  which  shall  include  the  following 
classes  of  tests: 

a.  Preshakedown  Tests.  This  portion  shall  contain  instructions  for  making 
physical  inspections,  turn-on  -off  procedures,  alignment,  adjustment,  and 
measurements  required  to  be  accomplished  prior  to  the  start  of  shakedown 
testing  of  the  facility. 

b.  Shakedown  Tests.  This  portion  shall  include  a  tabulation  of  the  equipment 
and  components  to  bo  subjected  to  shakedown  tests,  the  total  population  of 
electronic  parts  in  each,  and  the  required  duration  of  the  test  for  each. 

c.  Operational  Tests.  This  portion  shall  include  instructions  for  performing 
the  tests  required  to  demonstrate  that  all  equipment  programs  or  facilities 
covered  in  the  test  plan  and  work  specification  aro  properly  installed  and 
are  capable  of  performing  their  operational  mission  up  to  the  prescribed 
interfaces  with  other  portions  of  the  subsystem  and  system. 

VI.  WEIGHT  AND  MOMENT 

Include  the  following  when  determining  weight  and  moment  characteristics: 
total  weight,  weight  reference  to  baseline,  weight  reference  to  midpoint,  net  moment, 
and  weight  reference  to  centerline  (see  SUPSHIPINST  9290. 1C). 

VII.  TRAINING 

The  immediate  and  long-range  training  requirements  for  both  civilian  and 
military  personnel  must  bo  considered.  State  the  following  requirements: 

1.  Training  courses  required  including  course  length. 

2.  Training  materials  and  facilities  required. 

3.  A  summary  of  technical  data  required. 

4.  Instruction  advisory  services  required. 
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VIII.  MANPOWER 


When  th8  accomplishment  of  an  alteration  affects  any  aspect  of  manning,  the 
Ship  Manning  Document  (SMD)  must  be  updated  to  reflect  the  resulting  changes. 
Items  such  as  the  addition,  removal,  or  modification  of  equipment  and  systems  may 
necessitate  increases  or  decreases  in  existing  manning  levels,  and  must  be  considered 
in  view  of  all  manning  factors. 

IX.  PERSONNEL 

Number  and  type  to  install,  check  out,  operate,  and  maintain. 

X.  SPARES 

Number  and  type  of  spares.  For  permanent  installations,  required  changes  to 
the  Coordinated  Ships  Allowance  List  (COSAL). 


i 
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XIX.  TEST  AND  EVALUATION 


BACKGROUND 


DoD  Directive  5000.1  establishes  a  “fly  before  buy”  philosophy  for  all  mqjor 
acquisition  programs  (over  $50  million  RDT&E  and/or  over  $200  million  in  produc¬ 
tion);  this  policy  is  extended  by  SECNAV  Instruction  5000.1  to  all  Navy  acquisition 
programs.  This  has  resulted  in  revisions  to  the  policies  for  granting  service  approval 
making  Approval  for  Production  (AFP)  contingent  upon  a  satisfactory  completion  of  a 
test  and  evaluation  program  executed  by  COMOPTEVFOR  who  has  been  assigned 
responsibilities  as  the  Navy’s  independent  test  agency  (see  chapter  VII,  Approval  for 
Production).  Service  approval  is  a  primary  milestone  in  satisfying  an  operational 
requirement;  however,  each  of  the  steps  in  the  acquisition  cycle  has  test  and  evalu¬ 
ation  (T&E)  needs  to  be  met  even  if  no  mandatory  policy  exists. 


PURPOSES 


There  are  seven  basic  purposes  for  executing  a  T&E  program  defined  as 
follows: 


1.  Investigation.  Examines  natural  or  special  phenomena  in  an  operational 
environment  or  gathers  data  to  determine  preferred  technical  alternatives. 

2.  Diagnosis.  Determines  the  origins  and  nature  of  undesirable  behavior  ob¬ 
served  or  anticipated  in  a  test  item. 

3.  Evaluation.  Appraises  the  parameters  and  attributes  of  a  test  item. 

4.  Verification.  Confirms  tho  achievement  of  an  established  level  of  operation, 
the  suitability  of  interfacing  facets  (space,  fixtures,  power,  etc.),  or  the  ade¬ 
quacy  of  descriptive  item  documentation. 

5.  Qualification.  Proves  the  capability  of  the  test  item  of  meeting  established 
requirements. 

6.  Acceptance.  Determines  conformance  to  specification  requirements  prior '  r 
acceptance. 

7.  Appraisal.  Determines  optimum  procedures  and  tactics,  confirms  correcteu 
discrepancies,  and  verifies  suspected  discrepancies. 


Each  phase  of  the  acquisition  cycle  may  draw  on  one  or  more  test  purposes.  Investi¬ 
gations  are  used  to  support  the  Requirements  Definition,  Concept  Formulation,  Vali¬ 
dation,  and  Screening  phases.  In  tho  early  phases  the  investigation  serves  to  better 
define  the  operational  and  technical  requirements;  in  later  phases  it  serves  the  pur¬ 
pose  of  selecting  technical  alternatives.  Appraisals  may  be  performed  on  production 
equipments  to  verify  the  existence  or  correction  of  discrepancies  noted  in  operationi '. 
evaluation  (OPEVAL)  or  discovered  in  fleet  operations.  Appraisals  are  also  conducted 
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to  determine  optimum  procedures  and  tactics  for  utilizing  new  capabilities  or  adapt¬ 
ing  existing  capabilities  to  new  threats;  in  this  latter  vein,  they  may  also  be  con¬ 
ducted  to  better  define  and  determine  requirements  for  future  system  acquisitions. 
Since  appraisals  aro  not  part  of  any  particular  acquisition  program  because  they  are 
conducted  directly  in  support  of  CNO,  they  are  not  discussed  further  here.  The  other 
test  purposes  serve  the  program  phases  shown  in  table  XIX-1.  Notice  that  diagnostic 
tests  are  only  performed  in  conjunction  with  other  types  of  tests;  this  type  of  test 
serves  to  provide  the  analysis  data  to  support  corrective  engineering  efforts.  When 
structuring  a  test  program,  certain  elements  must  be  present  to  accomplish  the  pur¬ 
poses  of  the  testing,  these  elements  are  discussed  separately  for  each  purpose  later  in 
this  chapter. 


INSTRUCTIONS 


In  planning  for  test  and  evaluation,  there  are  two  vital  instructions: 

Test  and  Evaluation  —  OPNAVINST  3960.10. 

Mission  and  functions  of  Operational  Test  and  Evaluation  Force 

(OPTEVFOR)  -  OPNAVINST  5440.47  (series). 

OPNAVINST  3960.10  defines  the  types  of  priorities  and  services  available,  sets  forth 
procedures  for  the  prosecution  of  test  and  evaluation  programs,  establishes  Navy  test 
and  evaluation  policies,  encloses  milestone  checklists  for  T&E  planning,  and  imple¬ 
ments  the  policies  of  higher  authority  through  the  Test  and  Evaluation  Master  Plan 
(TEMP)  (diroctions  for  preparing  the  TEMP  are  included).  OPNAVINST  5440.47  is 
usoful  in  understanding  the  functions  of  COMOPTEVFOR.  The  3960  series  instruc¬ 
tions  provide  considerable  guidance  in  the  establishment,  planning,  and  implementa¬ 
tion  of  T&E  programs;  OPNAVINST  3960.10  should  be  followed  in  requesting  formal 
status  to  obtain  COMOPTEVFOR  assistance  and  to  request  fleet  services.  The  follow¬ 
ing  portions  of  this  chapter  are  intended  to  amplify  these  instructions  and  to  provide 
more  detailed  guidance  in  structuring  the  TEMP. 


GENERAL  PLANNING  FOR  EACH  PHASE 


When  program  planning  is  initiated,  T&E  plans  should  be  an  integral  part  of 
the  program  plans.  There  are  three  types  of  plans  for  T&E  —  the  TEMP,  the  master 
program  test  plan,  and  the  detailed  test  plan.  The  TEMP  should  be  established  as 
early  as  possible  since  it  is  the  planning  summary  which  serves  as  a  contact  with  the 
“outside  world."  The  program  T&E  plan  should  always  be  more  detailed  than  the 
TEMP  itself.  The  master  program  T&E  plan  should  initially  outline  the  purposes, 
scope,  and  objectives  of  the  T&E  intended  for  each  program  phase.  Referring  to  table 
XIX-1,  testing  occurs  at  several  levels  of  complexity  in  each  phase;  it  is  useful  to 
structure  the  master  program  test  plan  to  conform  to  the  Work  Breakdown  Structure 
(WBS)  (see  chapter  III,  Program  Planning)  so  the  individual  test  sequences  can  be 
scheduled,  costed  out,  coordinated,  and  tracked  at  each  level.  The  master  program 
test  plan  will  include  the  schedules  for  requesting  services,  for  preparing  test 
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facilities,  for  submitting  and  updating  the  TEMP,  and  for  executing  detailed  test 
plans  existing  for  each  cell  of  the  WBS,  additionally,  the  scope  and  objectives  will  be 
continually  updated  within  the  master  program  test  plan.  The  detailed  test  plans  are 
formulated  to  conform  to  the  master  plan  prior  to  the  initiation  of  the  test  phase  to 
which  they  apply;  they  contain  the  characteristics  to  be  tested  for;  the  test  procedures 
to  be  followed;  a  description  of  the  test  setups,  facilities,  and  test  equipments  needed, 
and  the  documentation  to  be  generated  in  support  of  the  T&E  effort. 

The  TEMP  and  the  master  program  test  plan  should  be  similar  in  content  and 
different  primarily  in  format.  Generally,  the  master  program  test  plan  will  be  ex¬ 
tended  to  a  lower  level  of  the  WBS  than  the  TEMP.  It  will  also  contain  cost  estimates 
for  each  detailed  test  and  will  list  the  alternative  plans  supporting  backup  ap¬ 
proaches.  Both  documents  are  the  responsibility  of  the  goverr-.  ent  project  manager. 

The  detailed  test  plans  are  normally  written  by  the  executing  activity, 
whether  in-house  or  contractor,  and  submitted  to  the  government  project  manager  for 
approval.  The  detailed  tests  respond  to  the  applicable  portions  of  system  specifica- 
tion(s).  To  check  the  accuracy  of  a  detailed  test  plan,  the  plan  is  compared  to  the  sec¬ 
tion  4  provisions  of  the  specification,  which  should  contain  all  the  inspections  and 
tests  to  be  performed  to  ensure  conformance  to  the  specification  requirements  (sec¬ 
tion  3).  The  Documentation  section  (chapter  XX)  contains  a  list  of  suggested  data 
item  descriptions  (DIDS)  to  be  used  in  requiring  test  plans  and  test  reports  in  a  con¬ 
tract  data  requirements  list  (CDRL).  A  test  report  is  comprised  of  the  detailed  test 
plan  plus  test  results  consisting  of  a  summation  and  analysis  and  of  test  data.  Most 
formal  test  reports  are  prepared  in  accordance  with  MIL-STD-831. 

The  specifications  quality  assurance/conformance  provisions  must  detail  tho 
following  information: 

Parameters  and  attributes  to  be  confirmed;  accept/reject  criteria 

Environmental  conditions 

Environmental  stress  levels 

Extent  of  testing 

Standard  test  methods  and  procedures  to  bo  employed 

Data,  analyses,  and  reports  required  to  demonstrate  conformance 

Test  requirements  originating  from  other  sources  such  as  ILS  plans  must  provide  this 
same  information.  Table  XIX-2  shows  typical  environmental  test  requirements.  The 
parameters  and  attributes  should  include  all  the  form,  fit,  and  function  characteris¬ 
tics  which  aro  required;  the  accept/rejoct  criteria  should  be  based  upon  the  specified 
parameter  tolerances  and  clearly  defined  attributes.  The  accept/rojcct  criteria  will 
normally  change  as  a  function  of  environmental  stress  level;  the  stress  levels  are: 


Operational 
Simulated  Operational 
Overstressed 

Uncontrolled 
Combined  Environment 


The  actual  application  environment 
Laboratory-controlled  environment 
Allowed  excursions  beyond  the  specified 
normal  conditions 

Laboratory  bench/ambient  environment 
Simulated  multiple  environments 
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Table  X1X-2.  Equipment  environmental  tests  and  requirements. 

General  Requirement* 

Desig¬ 
nator  Environment 

1  Temperature  (operating  and  nonoperating) 

2  Temperature-altitude 

3  Humidity 

4  Thermal  shock 

5  Vibration 

6  High-impact  shock 

7  Transport  shock 

8  Repetitive  shock 

9  EMC  (interference  and  susceptibility) 

10  EMP 

1 1  Electrical  transients  (voltage  and  frequency/long  term  and  short  term) 

12  lightning 

13  Magnetic  Held 

14  Acoustic  noise  (airborne  and  structure  borne) 

15  Inclination 

16  Radiation 

17  Nuclear  air  blast 

18  Gun  blast 

19  Wind 

20  Icing  with  wind 

21  Rain  and  anow,  snow  loading 

22  Sunshine 

23  Degree  of  enclosure 

24  Dust 

25  Salt  fog,  spray,  solution 

26  Damaging  (corrosive)  atmospheres 

27  Explosive  atmosphere 

28  Fungus 

29  Maintawability/bench  handling 

30  Reliability  (burn-in,  confidence,  indexing,  accelerated  life,  failure-mode  analysis) 

31  Combined -environment  testing  (temperature-humidity-vibration-electncal  transients  on/off  cycling) 

32  Oft/off  cycling 

33  Acoustic  susceptibility  (in  hi  th-noise  environments) 

34  Water  impact/hydroetatic  pressure 

35  Underwater  explosion  (for  hull-mounted  equipments  only) 

36  Drop  teat 

37  Equipment  special  environments 


Special  Requirement  Categories 


Gaiasu  EnYirtnagnto 


Kakt 


Vital  equipments 
Semi  vital  equipments 
Nonvital  equipments 
Exposed  equipments 
Sheltered  equipments 
Standard  requirements 


10,16,17  16  and  17  for  exposed  equipments  operating 

6  nonoperating  (normal  operating  before  and  after  shock) 

6  safety  criteria 

12,18,19,20,21,22.23,24 
23 

1 ,3, 5, 6, 9, 1 1 , 14 ,29,30,3 1 ,32,37 
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Table  XIX-2.  Equipment  environmental  tests  and  requirements  (continued). 
APPLICATION  REQUIREMENTS' 


Application 

Environment* 

General  Equipment  Spec 

Test  Spec 

Notes 

Shipboard 

8,13,15,25,26, 

27,33,34,35 

MIL-F.-16400 

MIL-E-16400 

for  (5)  MHrSTD-167 
for  (6)  MIL-S-901 
Environmental  Interfaces 
MIL-STD-1390  for  (15)  per 
MIL-E-16400  except  30* 
(operating)  and  60*  (without 
damage  or  spillage)  for  sub¬ 
marines 

Shore  (fixed) 

S3 

MIL-E-16400 

MIL-E-16400 

for  (5)  0.5  g  to  50  Hz 
for  (6)  MIL-S-S01  may  be 
waived 

Shore  (ruobile), 
transportable, 
vehicular) 

2,4,7.15 

MIL-E-16400 

MIL-STD-810 

MIL-STD-169 

MIL-STD-170 

MII^STD-210 

MIL-STD-1474 

MIL-D-1S570 

Airborne 

2,4,7,26,27,28 

MIL-E-5400  Prop,  Jet, 
and  Helo  Aircraft 
MIL-E-8189  Missiles, 
Boosters,  and  Allied 
Vehides 

MIL-E-11991  Guided 
Missiles 

MIL-STD-810 

MIL-T-5422 

(Navy) 

Gen  Equip  Spec 

Portable 

36  plus  applic¬ 
able  general 
application 

Same  as  general  applica¬ 
tion 

Same  as  general 
application  plui 
detail  spec 

Space 

2.4,7.8.16.33 

MIL-STD-1540 

MIL-STD-1540 

for  (9)  MIL-STD-1541  equip¬ 
ments  are  all  considered 
vital 

Test  equip¬ 
ment 

7,24,25,27,28, 

33 

MIL-T-28800 

MIL-STD-810 

Amphibious 

shipboard  and  shore  (mobile) 
combined 

Torpedoes 

22,24,25,28,34 

MIL-T- 18404 

MIL-T- 18404 

(37)  indudes  acceleration 

Shipboard  fire 
control 

2,8,13,15,25,26, 

27,33 

MIL-F-18870 

MIL-T- 18870 

same  as  Shipboard  except  as 
modified  by  MIL-T-18870. 

(2)  is  nonoperating  trans¬ 
portation  test 


'Environmental  requirements  should  consider  standard,  sheltered  or  exposed,  and  vital-aemivital-nonvital 
requirements  in  addition  to  those  listed. 
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Most  preliminary  evaluation  testing  is  performed  in  an  uncontrolled  environment; 
final  evaluation  testing,  acceptance  testing,  and  verification  testing  utilize  simulated 
operational  environmental  levels.  Simulated  environments  may  be  tested  singly,  dou¬ 
bly,  or  in  multiple.  Single  simulations  are  normally  used  because  diagnosis  test  data 
may  be  simultaneously  extracted  and  readily  analyzed  to  isolate  the  origins  of  unde¬ 
sirable  behaviors  and  the  causes  of  test  failures.  Double  simulations  are  used  when 
two  environmental  factors  are  very  closely  related,  such  as  temperature  and  humid¬ 
ity.  The  combined  environment  test  (CET)  is  a  multiple  simulation  intended  to 
closely  resemble  the  actual  operational  environmental  extremes.  Temperature, 
humidity,  vibration,  electrical  transients  (voltage  and  frequency),  and  RFT  environ¬ 
ments  can  be  economically  tested  together;  since  these  environments  may  produce 
Synergistic  effocts  in  combination  which  are  not  detectable  in  single-environment 
tests,  CET  is  an  essential  tool  for  preliminary  qualification  testing  (as  for  the  TEL- 
GAM  screen)  and  for  developing  confidence  that  equipments  will  operate  satisfacto¬ 
rily  in  the  operational  environments  (ref.  NELC  Technical  Report  1605).  CET  is  not 
normally  applicable  for  test  phases  employing  diagnosis  because  fault  data  cannot  be 
easily  traced  to  the  environmentally  susceptible  portion  of  the  design;  therefore,  CET 
is  normally  preceded  by  sequential  single-environment  testing  which  will  often  utilize 
overstress  parameters.  When  synergistic  faults  do  occur,  a  multiple  linear  regression 
model  can  be  employed  to  break  down  the  environmental  performance  of  the  equip¬ 
ment;  since  these  models  require  many  data  points  and  computer  support,  it  is  usu¬ 
ally  more  economical  to  rely,  at  least  initially,  on  previous  single-environment  results 
to  try  to  discover  weak  design  "oints.  All  final  qualification  testing  should  utilize  an 
operational  environment.  In  addition,  evaluation  testing  (especially  of  ADMs)  of 
equipments  which  have  critical  interfaces  (either  operationally  or  functionally)  with 
the  ultimate  platform  should  be  performed  in  the  operational  environment;  table 
XIX-3  illustrates  the  different  floet  sorvices  which  may  be  requested  for  such  testing. 
Requests  must  be  made  in  accordance  with  OPNAVINST  3960.10. 

The  confidence  attainable  through  testing  relies  on  four  factors: 

Accurate  specifications  of  the  test  parameters  and  the  environmental  condi¬ 
tions 

Acccpt/rejcct  criteria 

Extent  of  testing 

Test  methodology 

The  accuracy  of  the  specification  is  primarily  a  function  of  the  information  available 
to  put  into  it.  Good  records  of  prior  work  on  the  program,  good  test  data  practices, 
research  of  related  investigations  (see  chapter  IV,  Conceptual  Phase,  Gathering  Infor¬ 
mation),  and  previous  experience  with  similar  existing  equipments  (see  chapter  IV, 
Gathering  Information;  and  chapter  VII,  Initiating  Support,  Monitoring  the  Perform¬ 
ance  of  Equipments  in  the  Fleet)  will  contribute  to  accurate  parameter  specifications; 
table  XIX-3  and  appendix  A  are  provided  to  more  precisely  determine  environmental 
conditions.  The  accept/reject  criteria  can  be  manipulated  with  respect  to  allowed 
parameter  tolerances  to  yield  bigger  confidence  results.  As  the  environmental  stress 
level  is  eased  from  operational  to  simulated  to  uncontrolled,  the  accept/reject  criteria 
should  normally  be  tightened  with  respect  to  the  specified  tolerances  to  compensate 
for  the  lost  test  accuracy.  While  common  sense  and  knowledge  of  the  design  may  tem¬ 
per  the  final  determinations,  a  rule-of-thumb  is  to  tighten  the  criteria  by  10%  for 
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Table  X1A-3.  Fleet  services  which  may  be  requested  through 
COMOPTEVFOR  to  support  test  programs. 

Fleet  Research  Investigation  An  examination  of  natural  or  special  phenomena  in 

operational  environment,  required  by  a  develop¬ 
ment  agency  (DA)  in  the  prosecution  of  research 
and  for  which  the  assistance  of  operating  forces  is 
required.  The  research  investigation  is  not  neces¬ 
sarily  program  oriented,  but  is  primarily  in  the 
pursuit  of  basic  research  to  provide  a  continuing 
data  base  which  may  have  potential  application  in 
areas  of  naval  interst.  The  DA  is  responsible  for 
the  planning  of  Fleet  Research  Investigations,  and 
further  advises  the  operational  commander  pro¬ 
viding  the  support.  COMOPTEVFOR  shall  make 
arrangements  for  needed  fleet  support,  as 
requested  by  the  DA. 

Development:  Assist  Projects  to  provide  fleet  support  for  the  test  needed 

in  gathering  data  to  determine  the  direction  in 
which  development  should  proceed,  in  response  to 
a  requirement  during  the  conceptual  phase  of  a 
program.  They  may  also  relate  to  material 
improvements  of  equipments  already  in  the  Fleet. 
(Examples:  proposed  SHIPALTS,  ORDALTS,  Ser¬ 
vice  changes.) 

Several  such  projects  may  be  requested  to  provide 
sufficient  data  for  the  preparation  of  a  DCP  or 
comparable  document.  The  Development  Assist 
tests  are  primarily  the  responsibility  of  the  DA 
who  plans  the  tests.  The  test  plan  promulgated 
by  the  DA  will  be  mutually  agreeable  with 
COMOPTEVFOR  and  the  DA  with  respect  to  fleet 
unit  participation. 

QpnrntionaL Assist  An  Operational  Assist  project  is  assigned  by  CNO 

in  response  to  a  favorable  program  initiation  deci¬ 
sion  by  SECDEF  or  comparable  CNO  decision.  In 
addition,  a  DA  may  request  an  Operational  Assist 
project  for  material  improvement  programs  and  for 
certain  acquisition  programs  by  submission  of  a 
project  request.  This  project  is  primarily  the 
responsibility  of  the  DA,  and  its  major  purpose  is 
to  establish  confidence  in  the  program  worth 
and  readiness  for  the  commitment  of  resources  for 
full-scale  development. 
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Table  XIX-S.  Fleet  services  which  may  be  requested  through 
COMOPTEVFOR  to  support  test  programs  (continued). 

iuation  Project  A  TECHEVAL  is  assigned  by  CNO  in  response  to  a 
favorable  program  full-scale  development  decision 
by  SECDEF  or  comparable  CNO/CNM  decision 
approving  the  commitment  of  resources  for  full- 
scale  development.  In  addition,  a  TECHEVAL 
may  be  requested  by  a  DA  to  determine  suit¬ 
ability  for  other  acquisition  programs  and  mate¬ 
rial  improvement  including  conversions,  mqjor 
modifications,  and  modernizations.  In  the  case  of 
aircraft  or  missile  programs,  the  TECHEVAL  will 
include  the  Navy  Preliminaiy  Evaluation  (NPE)  or 
the  Navy  Technical  Evaluation  (NTE). 

A  TECHEVAL  is  performed  for  the  purpose  of 
investigating  systems  or  equipments  and  collecting 
information  which  will  aid  in  answering  technical 
questions  and  issues.  The  testing  and  analysis  con¬ 
ducted  by  or  for  the  DA  during  the  time  span  of  the 
TECHEVAL  project  are  to  permit  the  DA  to 
determine  whether  the  system  or  equipment  is 
functioning  in  a  technically  acceptable  manner, 
meets  design  and  technical  performance  specifi¬ 
cations,  and  is  technically  suitable  for  Operational 
Evaluation  (OPEVAL). 

The  DA  has  primary  responsibility  for  planning  the 
test  program,  including  the  coordinated  opera¬ 
tional  inputs  of  COMOPTEVFOR.  TECHEVAL 
and  OPEVAL  are  complementary  test  programs 
and  complete  initial  operational  test  and  evalu¬ 
ation.  Together  they  generate  data  and  address  the 
spectrum  of  questions  and  issues  to  be  considered 
prior  to  a  major  production  decision.  Accordingly, 
through  close  liaison,  COMOPTEVFOR  and  the 
DA  shall  ensure  that  the  test  plans  are  mutually 
agreeable  and  integrated  to  the  extent  that  they 
adequately  address  the  critical  questions  and 
issues  posed  in  the  governing  DCP  or  comparable 
document,  and  provide  for  the  maximum  practica¬ 
ble  use  of  common  test  data. 

Testing  during  TECHEVAL  may  be  conducted 
using  production  prototype  or  pilot  production 
models.  Prior  to  OPEVAL,  the  DA  shall  institute 
a  design  freeze  on  the  equipment  or  system  and 
certify  it  to  CNO/COMOPTEVFOR  as  ready  for 
OPEVAL. 


XIX-9 


Table  aLX-3.  Fleet  services  which  may  be  requested  through 
COMOPTEVFOR  to  support  test  programs  (continued). 

Evaluation  At  the  time  the  TECHEVAL  is  assigned  by  CNO, 

an  OPEVAL  will  also  be  assigned,  primarily  for 
planning  and  scheduling  purposes.  Project  opera¬ 
tions  under  OPEVAL  will  not  commence  until  the 
DA  has  certified  the  equipment.  The  project  is  the 
responsibility  of  COMOPTEVFOR  with  the 
development  agency  assisting  as  required; 
COMOPTEVFOR  collects  data  of  reasonable  scope 
and  depth  to  aid  in  the  decision-making  process  at 
program  milestones. 

The  primary  objectives  of  OPEVAL  are  to  ascertain 
that: 

The  system  or  equipment  functions  in  an  opera¬ 
tionally  satisfactory  manner  and  performs  reliably 
and  effectively  in  accordance  with  program  objec- 
ives  in  realistic  operational  conditions. 

The  system  can  bo  effectively  operated  and  main¬ 
tained  by  the  level  of  personnel  skill  anticipated  to 
be  available  under  service  conditions. 

There  is  reasonable  indication  that  logistic  sup- 
portability  in  a  deployed  status  is  feasible. 

All  test  questions  germane  to  a  production  decision 
are  adequately  examined. 

onal  Appraisal  The  assignment  of  a  Fleet  Operational  Appraisal 
project  by  CNO  may  be  initiated  by  a  recommenda¬ 
tion  submitted  by  COMOPTEVFOR  on  completion 
of  an  OPEVAL  or  by  request  of  a  Fleet  Commander 
in  Chief  or  type  commander.  The  purpose  of  this 
type  project  normally  is  to  provide  for  follow-on 
operational  test  and  evaluation,  to  develop  opti¬ 
mum  procedures  and  tactics  for  existent  systoms  or 
equipment  both  new  and  existent  in  the  Fleet,  to 
verify  the  performance  of  production  units,  and/or 
to  confirm  the  correction  of  discrepancies  pre¬ 
viously  disclosed.  It  also  may  be  used  to  examine 
and/or  develop  concepts  and  procedures  to  better 
define  and  determine  requirements  for  future  sys¬ 
tems  development. 
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each  easing  of  stress  level  and  by  5%  for  each  test  phase  precoding  NTE/OPEVAL. 
The  extent  of  testing  involves  the  length  of  testing  and  the  number  of  units  subjected 
to  test;  higher  confidences  may  be  obtained  with  more  test  time  and  more  guidance 
on  the  test  time  and  equipment  quantities  which  are  required  to  achieve  various  lev¬ 
els  of  confidence.  The  standard  test  methodologies  should  be  utilized  wherever  appli¬ 
cable,  as  modified  by  the  application  environment,  since  standard  test  procedures, 
test  equipment,  and  test  facilities  are  available  and  a  wealth  of  experience  with  the 
methodologies  has  been  developed  over  the  years;  table  XIX-4  lists  the  more  impor¬ 
tant  test  method  standards.  Parameters  which  lend  themselves  to  statistical  analysis 
may  be  tested  satisfactorily  with  a  sampling  procedure;  others  dependent  on  nonvary¬ 
ing  design  features  require  the  testing  of  all  test  items.  Each  parameter  should  be 
reviewed  with  sampling  in  mind  when  large  quantities  of  test  items  are  involved, 
since  test  costs  can  be  significantly  roduced  by  sampling  procedures.  Different  num¬ 
bers  of  test  items  may  be  used  for  various  tests;  generally,  the  test  item  population 
will  be  divided  up,  with  some  tests  performed  on  the  entire  population  and  different 
tests  run  on  the  various  portions  of  the  population. 

Overstressed  environments  may  apply  to  either  operational  or  simulated  envi¬ 
ronmental  test  levels.  When  presont  in  operational  environments,  the  conditions  of 
overstress  and  allowable  degradation  of  the  equipment  should  be  included  in  the 
specification.  Overstressod  conditions  may  also  be  applied  in  simulated  environments 
to  (1)  accelerate  aging  factors,  (2)  discover  environmentally  susceptible  design  fea¬ 
tures,  and  (3)  increase  test  confidence.  While  very  useful,  the  stress  mechanisms 
should  bo  understood  as  to  how  they  should  affect  a  design  so  that  realistic  accep¬ 
tance  criteria  may  be  formulated.  The  nature  of  some  test  items  precludes  certain 
types  of  overstress  (paper  tape  cannot  be  tested  at  100%  humidity,  for  instanco). 
Where  the  stress  mechanisms  are  not  known,  overstresses  not  reflected  in  the  usago 
environment  should  be  avoided. 


ESTIMATING  COSTS  OF  T&E 


Estimating  T&E  costs  is  an  important  function  to  program  planning  (see 
chapter  III).  Early  detailed  T&E  master  planning  can  help  significantly;  nevertheless, 
there  are  many  uncertainties  for  which  contingency  plans  must  be  budgeted.  The  fol¬ 
lowing  list  of  factors  should  be  considered  in  T&E  cost  estimating; 

Facilities  (ships,  test  beds,  laboratories) 

Test  equipments 

Instrumentation 

Documentation  (detailed  test  procedures,  technical  manuals,  analyses,  reports, 
equipment  surveys,  etc.) 

Personnel  (types,  time  needed) 

Repair  capabilities  (including  technicians,  technical  manuals,  test  equipments, 
tools,  spare  parts) 
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Table  XIX-4.  Test  methodology  standards.* 

MIL-STD-105  Sampling  Procedures  and  Table  for  Inspection  by  Attributes 
MIL-STD-108  Definitions  of  and  Basic  Requirements  for  Enclosures  for  Elec¬ 

tric  and  Electronic  Equipment 

MIL-STD-167  Mechanical  Vibrations  of  Shipboard  Equipment 
MIL-STD-170  Moisture  Resistance  Test  Qycle  for  Ground  Signal  Equipment 
MIL-STD-220  Method  of  Insertion-loss  Measurements 

MIL-STD-271  Nondestructive  Testing  Requirements  for  Metals 
M1L-STD-4 14  Sampling  Procedures  and  Tables  for  Inspection  by  Variable  for 
Percent  Defective 

MIL-STD-446  Environmental  Requirements  for  Electronic  Parts 
MIL-STD-449  Radio  Frequency  Spectrum  Characteristics,  Measurement  of 
MIL-STD-462  Electromagnetic  Interference  Characteristics,  Measurement  of 

MIL-STD-469  Radar  Engineering  Design  Requirements  for  Electromagnetic 
Compatibility 

MIL-STD-471  Maintainability  Demonstration 

MIL-STD-690  Failure  Rato  Sampling  Plans  and  Procedures 
MIL- STD-740  Airborne  and  Structurebome  Noise  Measurements  and 

Acceptance  Criteria  of  Shipboard  Equipment 
MIL-STD-750  Test  Methods  for  Semiconductor  Devicos 
MIL-STD-781  Reliability  Tests 

MIL-STD-810  Environmental  Test  Methods 

MIL-STD-883  Tost  Mothods  and  Procedures  for  Microelectronics 

MIL-STD-1310  Shipboard  Bonding,  Grounding,  and  Other  Techniques  for  Elec¬ 
tromagnetic  Compatibility 

MIL-STD-13I1  Test  Metliods  for  Electron  Tubes 
MIL-STD-I344  Test  Mothods  for  Electrical  Connectors 
MIL-STD-1472  Human  Engineering  Design  Criteria  for  Military  Systems, 

Equipment,  and  Facilities 

MIL-S-901  Shock  Tests,  High  Impact;  Shipboard  Machinery,  Equipment, 

and  Systems,  Requirements  for 

M1L-B-5087  Bonding,  Electrical,  and  Lightning  Protection  for  Aerospace 

Systems 

M1L-T-5422  Testing,  Environmental,  Airborne  Electronic  and  Associated 

Equipment 

MIL-D-13570  Dust,  Testing  by  Exposure  to 


‘Additional  methods  aro  specified  by  the  various,  general  equipment  specifications 
(M1L-E-16400,  MIL-T-28800,  M1L-E-5400,  etc.)  and  by  military,  federal,  and  indus¬ 
trial  standards.  Refer  to  the  DoD  Index  of  Specifications  and  Standards  for  methods 
of  less  general  application  and  for  the  most  recent  editions  of  the  above  standards. 
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Failures  will  occur  during  testing,  depending  on  the  design  and  complexity  of  the 
equipment,  the  repair  functions  can  double  or  triple  the  T&E  period  and  increase 
costs  by  a  factor  of  2  to  10  (5  is  considered  a  good  planning  figure).  A  failure  to  meet 
the  test  objectives  is  much  more  probable  in  early  program  phases  than  later  if  the 
test  planning  —  the  entire  program  planning,  for  that  matter  —  is  done  properly. 
Therefore,  T&E  contingencies  should  be  made  larger  in  the  early  phases  relative  t£ 
the  estimated  “no  failure"  cost  than  in  later  phases.  The  T&E  schedules  should  also 
reflect  contingencies  and  possible  slippages,  and  the  program  plan  should  incorporate 
the  necessary  flexibility  to  overcome  these  uncertainties.  T&E  personnel  (such  as 
NOSC  Code  95)  should  be  consulted  in  structuring  the  T&E  portions  of  the  program 
plan,  in  formulating  the  various  test  documents,  and  in  scheduling  their  environ¬ 
mental  and  simulation  facilities. 


T&E  PLANNING 


It  is  essential  that  sufficient  developmental  and  operational  test  and  evalu¬ 
ation  be  accomplished  to  ensure  that  the  systems  introduced  into  service  meet  the 
defined  operational  requirements  and  are  suitable  for  service  use.  The  operational 
effectiveness  and  suitability  of  these  systems  must  be  determined  to  a  high  degree  of 
confidence  before  they  nro  approved.  Complete  and  realistic  programs  for  such  deter¬ 
minations  are,  therefore,  matters  of  concern  to  all  levels  of  management. 

Tho  Commander,  Operational  Test  and  Evaluation  Force  (COMOPTEVFOR), 
has  boen  assigned  responsibilities  as  an  independent  test  agency  for  the  required  op¬ 
erational  test  and  evaluation  of  now  systems.  The  objectives  of  COMOPTEVFOR  arc 
to  ensure  that  early  and  realistic  operational  tests  and  evaluations  are  planned;  to 
ensure  that  appropriate  initial  operational  tests  and  evaluations  are  conducted  prior 
to  schodulod  decision  milestones;  and  to  provide  an  independent  assessment  of  the 
operational  effectiveness  and  suitability  of  the  system  to  CNO. 

CNO  has  responsibility  for  ensuring  the  adequacy  of  the  Navy’s  overall  test 
and  evaluation  program.  T&E  policy  and  guidance  are  exorcised  tlirough  the  Direc¬ 
tor,  RDT&E  (OP-098),  and  in  accordance  with  overall  policies  established  by  the  Sec¬ 
retary  of  the  Navy.  Tho  Director’s  responsibilities  include: 

Reviewing  proposed  T&E  objectives  and  requirements  for  fleet  services  to  sup¬ 
port  T&E  programs. 

Assigning  all  T&E  projects  and  their  priorities  to  the  operating  forces  for 
prosecution. 

Receiving,  reviewing,  and  assessing  all  OT&E  reports  for  systems,  and  acting 
for  CNO,  as  the  sole  releasing  authority,  for  such  reports  and  information  to 
higher  authority. 

Ensuring  that  adequate  funding  for  all  required  and  approved  T&E  is  identi¬ 
fied  in  the  programming  and  budgetary  system. 
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TEST  AND  EVALUATION  OF  MAJOR  SYSTEMS 


Test  and  evaluation  of  systems  is  discussed  in  the  following  three  subsections. 

Developmental  Test  and  Evaluation  (DME) 

This  category  or  class  of  tests  and  evaluations  is  conducted  under  the  sponsor¬ 
ship  of  the  developing  agency,  and  is  undertaken  for  the  specific  purpose  of  facilitat¬ 
ing  the  evolution  of  a  system.  The  development  test  plan  must  be  structured  and 
executed  in  such  a  manner  as  to  ensure  the  generation  of  data  which  can  also  be  used 
to  make  the  required  assessments  of  the  operational  effectiveness  and  suitability  of 
the  emerging  design.  Test  programs  in  this  category  include  engineering  tests, 
contractor/laboratory  demonstrations,  naval  technical  evaluations,  and  Navy  prelimi¬ 
nary  evaluations  and  deficiency  correction  tests.  The  test  articles  required  by  the 
DT&E  programs  are  characterized  as  advanced  development,  engineering,  and  pro¬ 
duction  prototype  models. 

Operational  Test  and  Evaluation  (OME) 

This  category  of  testing  includes  all  test  and  evaluation  effort  participated  in, 
or  undertaken,  for  the  purpose  of  obtaining  operational  information  throughout  the 
life  cycle  of  the  system,  and  supports  both  the  acquisition  process  and  the  optimum 
employment  of  the  system.  It  is  accomplished  in  two  phases,  Initial  Operational  Test 
and  Evaluation  (IOT&E)  and  Follow-on  Operational  Test  and  Evaluation  (FOT&E). 

IOT&E  is  that  testing  and  evaluation  accomplished  by  or  under  the  supervi¬ 
sion  of  the  Navy's  independent  testing  agency  prior  to  the  first  major  production  de¬ 
cision.  IOT&E  reports  address  the  operational  effectiveness  and  suitability  (including 
reliability,  compatibility,  and  maintainability)  of  the  system,  including  the  critical 
issues  identified  in  the  DCP  or  comparable  acquisition  document  in  accordance  with 
defined  operational  requirements.  IOT&E  is  conducted  in  accordance  with  DCP. 

FOT&E  is  the  continuing  test  and  evaluation  of  a  system  conducted  under 
fleet  conditions  by  fleet  operational  personnel  using  initial  production  systems  for 
the  purposes  of  verifying  system  performance,  validating  correction  of  deficiencies 
previously  identified,  and  refining  tactical  employment  doctrine  and  requirements  for 
personnel  and  training. 

Acceptance  Trials 

The  Board  of  Inspection  and  Survey  (BIS)  is  responsible  to  CNO  for  conduct¬ 
ing  acceptance  trials  prior  to  Navy  acceptance  from  the  contractor.  The  Board 
inspects  the  material  condition,  requires  demonstration  of  equipments  and  systems  to 
ensure  performance  in  acco  dance  with  the  requirements  of  the  contract  specifica¬ 
tions,  and  determines  compliance  with  the  characteristics  established  oy  CNO.  After 
completion  of  acceptance  trials  the  Board  reports  its  findings  to  CNO,  identifying 
deficiencies  for  which  they  recommend  corrective  action  prior  to  delivery.  Upon  suc¬ 
cessful  completion  of  final  contract  trials,  the  Board  reports  to  CNO  its  recommenda 
tions  for  final  settlement  of  the  contract. 
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T&E  CYCLE 


The  usual  sequence  of  test  events  followed  during  the  acquisition  of  a  system 
is  illustrated  in  figure  XIX-1.  Those  mqjor  programs  subject  to  DSARC  review  shall 
adhere  to  the  following  procedures. 

During  the  drafting  of  the  initial  DCP,  the  CNO  development  and  program 
coordinators  shall  coordinate  with  the  cognizant  SYSCOM  or  development  agency 
project  manager  and  COMOPTEVFOR  in  preparing  the  statement  of  critical  ques¬ 
tions  and  issues  which  tests  will  address.  Plans  and  estimates  of  the  required  test  and 
evaluation  shall  be  incorporated  and  schedule  milestones  identified.  The  proposed  test 
and  evaluation  programs  and  schedules  will  be  reviewed  for  compliance  with  Navy 
policy  by  CNO  (OP-983). 


Figure  XIX-1.  Sequence  ot  acquisition  tost  events. 


During  the  advanced  development  or  validation  phase,  subsequent  to  DSARC 
I,  the  development  agency  and  OPTEVFOR  will  refine  the  critical  questions  and 
issues  to  be  addressed  by  T&E.  Prior  to  DSARC  II  COMOPTEVPOR  will  make  an 
assessment  of  the  operational  suitability  and  effectiveness,  based  upon  all  valid 
IOT&E  data  extracted.  The  applicable  portions  of  the  DCP  for  DSARC  II  will  be 
updated  with  this  additional  information  by  the  CNO  development  and  program  coor¬ 
dinators.  The  refined  critical  questions  and  issues  and  the  test  program  will  be  re¬ 
viewed  for  adequacy  and  compliance  with  Navy  T&E  policy  by  CNO  (OP-983).  The 
revised  DCP  will  reflect  the  status  of  developmental  testing  and  indicate  technical 
and  operational  risks  outstanding. 

Once  full-scale  development  has  been  authorized  by  SECDEF,  after  DSARC 
milestone  II,  DT&E  will  be  oriented  toward  resolving  the  applicable  critical  questions 
and  issues  and  achieving  a  system  that  is,  in  all  respects,  ready  for  production  and 
suitable  for  service  use.  Prior  to  DSARC  milestone  III  IOT&E  will  address  operation¬ 
ally  oriented  critical  questions  and  issues,  using  data  generated  in  the  DT&E  pro¬ 
grams,  BIS  trials,  and  operational  tests.  OPTEVFOR  will  gather  all  valid  IOT&E 
remits,  <jvt '  jate  them,  and  prepare  an  independent  assessment  of  system  operational 
effectiveness  and  suitability.  This  report  will  be  directly  forwarded  to  CNO.  CNO  will 
provide  an  assessment  of  the  test  results  in  terms  of  the  response  to  the  critical  ques¬ 
tions  and  issues  to  the  appropriate  OSD  offices. 

Follow-on  test  and  evaluation  will  be  conducted,  following  a  favorable  produc¬ 
tion  decision  as  directed  by  CNO.  Follow-on  OT&E  may  be  desirable  in  order  to  ver¬ 
ify  performance  of  the  new  production  units,  further  develop  doctrine  and  tactics,  and 
complete  the  evaluation  of  the  reliability,  maintainability,  and  logistic  supportability 
of  the  production  system. 

INSURV  conducts  acceptance  for  service  use  and  specifies  conditions  for 
acceptance  from  the  contractor.  Acceptance  for  service  use  is  required  prior  to  deliv¬ 
ery  by  the  contractor.  The  operational  tests  and  evaluations  by  OPTEVFOR  and  the 
acceptance  for  service  use  by  INSURV  are  mutually  supporting.  An  integrated  test 
program  is  desirable  to  prevent  duplication  and  to  make  available  to  both  OPTEV¬ 
FOR  and  INSURV  all  test  data  as  they  become  available. 


T&E  MANAGEMENT  OF  OTHER  THAN  MAJOR  SYSTEMS 


Programs  of  this  category  are  not  subject  to  the  DSARC  decision  processes. 
They  shall  be  subject  to  a  similar  review  and  decision  process  within  the  Department 
of  the  Navy.  CNO  monitors  these  programs  and  renders  the  productive  decision  after 
appropriate  developmental  and  operational  test  and  evaluation  of  the  new  system  has 
been  accomplished.  CNO  will  delegate  the  review  process  to  the  cognizant  SYSCOM. 
Normally,  CNO  will  exercise  the  review  and  decision  process  for  those  programs  hav¬ 
ing  an  RDT&E  cost  in  excess  of  $5  million,  or  a  production  cost  in  excess  of  $20  mil¬ 
lion.  However,  CNO  may  exercise  the  review  and  decision  process  for  programs  below 
these  thresholds  dependent  upon  the  mission  essentiality  of  the  program.  The  test 
and  evaluation  requirements  shall  parallel  those  established  for  mqjor  programs. 
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T&E  MANAGEMENT  OF  SHIP  CONSTRUCTION 


In  the  systems  acquisition  process,  ship  construction  is  a  unique  procedure. 
The  test  and  evaluation  for  mqjor  or  other  new  systems  in  a  ship  shall  be 
accomplished  with  cognizant  program.  Appropriate  information  will  be  provided  in 
the  Ship  Program  DCP  on  which  to  base  a  ship  production  decision  and  schedule  of 
specific  follow-on  management  reviews  for  the  ship  program. 

The  requirement  to  test  and  evaluate  a  system  prior  to  production  approval 
may  only  be  waived  by  the  Secretary  of  the  Navy. 

Mqjor  programs  require  SECDEF  approval. 

NUCLEAR  WEAPONS  T&E 


Joint  AEC-DoD  development  of  a  new  nuclear  weapon  is  governed  by  the  1953 
“Agreement  between  the  AEC  and  the  DoD  for  the  Development,  Production  and 
Standardization  of  Atomic  Weapons.”  Test,  evaluation,  and  acceptance  of  nuclear 
weapons  are  to  be  conducted  in  accordance  with  that  agreement  and  implementing 
DoD  issuances  and  Navy  instructions. 
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XX.  DOCUMENTATION 


XX.  DOCUMENTATION 


The  goal  of  the  government  with  respect  to  documentation  has  always  been  to 
acquire  only  the  data  essential  for  adequate  development,  procurement,  operation  and 
support  of  systems  and  equipment.  Yet  the  volume  and  cost  of  data  have  somehow 
grown  over  the  years  until  today  both  are  enormous.  DoD  is  therefore  obliged  in  the 
interest  of  meeting  its  goal  to  fight  a  constant  holding  battle  against  proliferation  of 
data  requirements.  Directive  after  directive  demands  the  exercise  of  judgment  and 
restraint  in  the  specification  of  data  items  for  procurement. 

The  problem  at  the  procuring  level  lies  in  determining  the  essential  minimum, 
which  is  by  no  means  necessarily  the  absolute  minimum.  Underordering  can  be  as 
costly  as  overordering.  The  manager  who  habitually  buys  only  contractor  drawings  is 
often  one  day  obligated  to  pay  sky-high  prices  for  data  which  must  be  ordered  on 
unfavorable  terms  after  the  original  contract  is  let.  (Furthermore,  once  pinched  by 
his  own  frugality,  such  a  manager  is  likely  to  err  in  the  opposite  direction  —  to  rou¬ 
tinely  order  all  the  documentation  for  which  a  need  may  conceivably  develop.) 

The  answer  to  the  problem  lies  in  management.  The  project  manager  must 
define  data  requirements  as  rigorously  as  equipment  requirements,  especially  when 
the  emphasis  is  on  low-cost  acquisition. 


PM  GUIDELINES 


The  project  manager  can  simplify  tho  documentation  task  by  obtaining  assis¬ 
tance  from  data  management  specialists  (Product  Assurance  Division,  at  NOSC),  by 
procuring  documentation  through  the  internal  organisations  established  for  thnt  pur¬ 
pose  (Technical  Information  Division,  at  NOSC),  and  by  observing  the  following  koy 
guidelines: 

1.  On  the  basis  of  project  requirements  and  the  end  use  of  the  related  system/ 
equipment,  and  as  early  as  possible,  establish  a  preliminary  shopping  list 
of  the  types  of  documonts  that  you,  as  the  developer,  and  tho  government, 
as  the  user,  conceivably  could  neod  to  permit  effective  decisions,  evalu¬ 
ations,  and  actions.  On  large,  complex  projects,  get  similar  inputs  from 
responsible  task  leaders.  Include  requirements  specified  by  the  sponsor. 

Use  tho  standards  in  table  XX-1  and  the  Acquisition  Management  Systems 
and  Data  Requirements  Control  List  (AMSDL)  to  identify  candidate  data 
items.  The  AMSDL  is  available  at  the  Data  Management  Office  (DMO)  or 
on  the  VSMF  service;  it  is  published  as  DoD  5010.12-L  twice  annually. 

2.  Rigorously  evaluate  the  need  for  each  data  item  on  the  shopping  list.  Who 
will  need  it?  When?  Why?  What  or  who  will  suffer  without  it?  How  much? 
Will  the  information  possibly  be  available  elsewhere  in  other  project  docu¬ 
mentation,  or  in  documents  already  in  the  government’s  possession?  In 
other  words,  determine  as  well  as  you  can  the  true  essentiality  of  each 
data  item.  Use  an  arbitrary  scale  of,  say,  five,  and  assign  a  value  to  each 
item. 


XX-1 


3.  Estimate  the  costs  of  engineering  and  publication  efforts  on  each  data 
item;  weigh  against  the  essentiality  value  given  the  item;  and  obtain  a 
relative  cost-effectiveness  figure  for  the  item. 

4.  Make  initial  decisions  for  or  against  inclusion  of  the  items  and  establish 
an  initial  “minimum  essential”  data  requirements  list.  When  a  contractor 
is  involved  and  the  total  contract  exceeds  $1  million,  or  when  contract 
data  will  cost  over  $100,000,  contract  data  requirements  must,  be  approved 
by  a  data  requirements  review  board  as  explained  in  NOSCINST  4000.1, 
Data  Management  Program.  The  Data  Management  Office  will  assist  in 
determining  the  appropriate  data  items  and  in  tailoring  the  Data  Item  De¬ 
scriptions. 

5.  Subject  the  initial  data  requirements  list  to  continuous  review  and  valida¬ 
tion  in  the  light  of  project  developments. 

6.  When  ordering  data  from  contractors,  specify  use  of  the  contractor’s  for¬ 
mat  if  it  is  suitablo  and  the  data  item  is  to  be  used  solely  within  the  gov¬ 
ernment.  Impose  MILSPEC  and  standard  content  and  format  requirements 
only  for  specifications,  drawings,  manuals,  etc.,  that  must  moet  very  strict 
needs  for  procurement,  operation,  and  support.  Manuals  should  be  pro¬ 
cured,  at  NOSC,  through  the  Technical  Manuals  section  of  the  Technical 
Information  Division. 

7.  On  RFPs,  encourage  contractors  to  submit  alternative  data  packages  to 
the  data  package  represented  by  the  Contract  Data  Requirements  List  (DD 
Form  1423).  Negotiate  in  favor  of  the  least-expensive  package  submitted 
which  satisfies  tho  government’s  needs  as  specified  in  the  statement  of 
work. 

8.  If  contractors  are  to  provide  data  items,  determine  as  early  as  possiblo 
when  tho  items  aro  needed  for  use.  Order  them  under  the  appropriate  con¬ 
tract  and  have  them  delivered  as  they  are  nooded.  These  practices  will  help 
to  assure  that  the  government  obtains  data  at  the  lowest  possible  cost, 
that  the  delivered  data  are  usable  rather  than  in  need  of  revision,  and  that 
government  storage  and  handling  are  minimized. 


TYPICAL  DATA  ITEMS 


Table  XX-1  contains  a  list  of  subjects  of  concern  to  project  managers  and  engi¬ 
neers.  Tho  subjects  to  be  dealt  with  will  vary  with  tho  project.  The  AMSDL  contains 
over  1000  Data  Item  Descriptions  (DIDs)  approved  for  use  in  37  broad  categories. 

Documents  covering  the  subjects  listed  in  table  XX-1  can  be  grouped  into  four 
broad  categories  according  to  their  purpose: 

Those  used  for  program  administration,  control,  and  historical  reference 

Those  describing  systems  and  equipment 


XX-2 


T title  Xi-1.  Sources  of  data  item  descriptions  for  programs. 


Configuration  Management 

Design  Reviews 
Drawings 

Electromagnetic  Compatibility 
Human  Engineering 
Safety 
Software 

Specifications 
Product  Assurance 
ILS 

Level  of  Repair 
Reliability 


Maintainability 
Technical  Manuals 


Training 

Testability 

Provisioning 


M1L-STD-480 

MIL-STD-481 

MIL-STD-1521 

DOD-D-IOOO 

MILrSTD-461 

DOD-HDBK-763 

MIL-STD-882 

MIL-STD-2167A 

MIL-STD-2168 

MIL-STD-490 

M1L-STD-2107 

M1L-STD-1702 

MIL-STD-1390 

MIL-STD-785 

MIL-STD-756 

MIL-STD-781 

MIL-STD-1629 

MIL-STD-2074 

MIL-STD-2155 

MIL-STD-1543 

MIL-STD-470 

MIL-STD-2080 

MIL-STD-1790 

MIL-STD-7298 

MIL-STD-82376 

MIL-STD-1379 

MIL-STD-2165 

MIL-STD-1345 

MIL-STD-1561 


Standardization/Parts  Control  Program  M1L-STD-965 

Packaging,  Handling,  and  Transportation  MIL-STD-2073-1 

MIL-P-9024 

NOTE:  Use  the  most  current  editions  of  each  standard  or  specification  cited.  After  deter¬ 

mining  which  data  items  are  required,  use  the  most  recent  edition  of  the  identified 
DID,  including  superseding  editions. 


XX-3 


Those  used  to  control  design  changes 

Those  required  to  support  design,  operation,  maintenance,  and  support 


The  primary  documents  in  each  category  are  discussed  briefly  in  the  following 
paragraphs.  Refer  to  Chapter  XXII  for  more  information  on  those  items  concerned 
with  program  control. 


SYSTEM/EQUIPMENT  DESCRIPTION 


Specifications 

Specifications  provide  narrative  details  of  performance  and  physical  character¬ 
istics  and  their  quality  assurance  requirements  for  systems,  equipments,  assemblies, 
subassemblies,  components,  etc.  Development-type  specifications  are  prepared  to 
establish  initial  baselino  configurations  for  developing  prototype  models.  Product- 
type  specifications  reflect  the  finally  developed  configurations  and  are  used  to  procure 
operational  models.  Specifications  are  prepared  to  commercial/industrial  or  military 
standards.  Program-peculiar  military  specifications  are  required  by  MIL-S-83490, 
which  covers  the  requirements  for  Forms  la,  lb,  2,  and  3  and  Types  A,  B,  C,  D,  and  E 
specifications.  The  detail  requirements  for  Form  la  specifications  of  all  typos  are  pre¬ 
sented  by  MIL-STD-490,  while  M1L-STD-961  provides  overall  requirements  for  mili¬ 
tary  specifications  in  general.  MIL-STD-962  provides  requirements  for  military 
standards  and  handbooks. 

An  engineer,  whether  he  is  a  circuit  designer  or  has  the  total  responsibility  for 
the  overall  system  design,  should  be  familiar  with  MIL-STD-490,  since  it  provides  the 
guidance  for  writing  specifications  which  will  be  followed  throughout  the  various 
stages  of  military  system  development.  A  specification  which  is  not  comprehensive 
(i.e.,  does  not  cover  such  requirements  as  reliability,  human  engineering,  support 
documentation,  maintainability,  or  logistics)  will  practically  guarantee  higher  sup¬ 
port  costs  during  a  system’s  scheduled  lifetime  and  may  also  shorten  the  duration  of 
that  lifetime. 

Specifications  document  system/equipment  requirements;  therefore,  they  form 
a  vital  link  in  the  systems  engineering  task  of  “requirements  flowdown  and 
traceability." 

Engineering  Drawings 

Engineering  drawings  are  the  designer's  primary  means  of  communication. 
They  may  be  prepared  in  accordance  with  commercial/industrial  or  military  stan¬ 
dards.  The  Department  of  Defense  recognizes  three  levels  of  drawings  based  on 
intended  use  (DoD-D-1000).  DoD-STD-100  provides  the  for.  *.  and  content  require¬ 
ments  for  the  4  dozen  or  so  drawing  types  which  may  be  requ  sd.  The  DoD-D-1000 
levels  define  the  depth  of  information  contained  in  a  drawing  and  determine  what 
kinds  of  DoD-STD-100  drawings  may  be  required. 
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Test  Procedures  and  Reports 


Procedures  and  reports  are  required  for  tests  of  specified  performance  and 
physical  characteristics  for  developmental  and  production  models.  They  are  prepared 
by  the  development  activity  and  manufacturer  in  accordance  with  their  own  or  the 
buyer’s  requirements.  Various  requirements  documents  of  the  several  military  serv¬ 
ices  and  agencies  address  the  content  and  format  of  test  procedures  and  reports  and 
may  be  imposed  on  the  preparer. 

In  general,  every  specification  requirement  has  a  corresponding  test  or  inspec¬ 
tion  procedure  which  will  be  reported  each  time  the  requirements  are  verified. 

Engineering  Reports 

During  equipment  development,  reports  are  often  required  by  the  purchaser 
on  such  engineering  aspects  as  reliability  and  maintainability  predictions  and 
demonstration,  human  engineering,  and  safety.  The  specialty  engineers  involved  pre¬ 
pare  such  reports  to  their  own  in-house  requirements  or  may,  in  the  case  of  items  for 
the  military,  have  to  comply  with  applicable  military  specifications  and  standards. 

Many  detailed  specification  requirements  and  design  characteristics  result 
from  engineering  analyses  rather  than  operational  requirements.  The  systems  engi¬ 
neer  uses  engineering  reports  to  document  further  refinements  to  the  design  baseline 
and  to  ensure  consistency  between  designer  decisions  and  system  design  require¬ 
ments. 

Engineer’s  Notebook 

An  engineer’s  notebook  should  bo  established  the  moment  a  now  design  is 
begun.  Instructions,  dates,  findings,  decisions,  reasons,  conclusions,  etc.,  should  be 
included.  A  notebook’s  value  varies  with  the  circumstances  and  the  extent  of  its  con¬ 
tents.  Potential  benefits  of  a  notebook  include  patent  rights,  applications  to  future 
projects,  defense  of  actions  taken,  information  for  new  personnel,  and  general  future 
reference. 

DESIGN  CHANGE  CONTROL 


Equipment  configuration  management  is  a  normal  activity  of  most  companies 
and  government  developers.  Effective  configuration-change  control  employs  a  system 
of  recorded  change  data  subject  to  review  and  approval.  The  design  engineer  provides 
the  technical  inputs  for  this  documentation,  the  systems  engineer  is  responsible  for 
approving/rqjecting  proposed  changes.  Quality  engineers  and  configuration  managers 
track  approved  changes  and  assure  their  correct  incorporation  into  the  design. 

Engineering  Change  Proposals  (ECPs) 

Configuration  changes  are  formally  initiated  by  a  change-proposal  form/docu- 
ment  which  must  be  reviewed  and  approved  before  the  subject  changed)  can  be  incor¬ 
porated  in  the  design  or  equipment.  The  designer  himself  may  initiate  the  proposal  or 
may  cooperate  with  another  engineer  who  has  the  original  idea  for  a  change.  In 
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either  case,  the  designer  must  describe  how  the  proposed  change  affects  his  area  of 
design  responsibility  by  completing  the  change  proposal  form/document  and  including 
appropriate  drawing  and  specification  (see  SPECIFICATION  CHANGE  NOTICES) 
revisions  for  reference  as  well  as  required  costs  and  schedule  revision  information. 

The  military  services  must  follow,  and  cite  to  contractors,  the  requirements  of  MIL- 
STD-480  or  MIL-STD-481,  which  employ  DD  Form  1692  or  1693,  •  espectively,  as  the 
engineering  change  proposal  document.  Variations  of  these  forms  may  be  used  by 
industry  and  DoD  for  in-house-only  applications.  The  completed  form/document  is 
reviewed  by  a  configuration  control  board  which  may  immediately  decide  for  or 
against  the  proposal  or  return  it  to  the  originators)  with  a  request  for  additional 
information.  A  signed,  approved  ECP  constitutes  authorization  to  proceed  with  the 
change,  a  process  that  normally  includes  the  preparation  of  engineering  change 
orders  and  specification  change  notices  (see  following). 

Engineering  Change  Orders 

Properly  authorized  engineering  change  orders  provide  the  technical  and  pro¬ 
cedural  information  necessary  to  fully  implement  the  authorized  change  to  the 
design/equipment  and  to  the  engineering  documentation  that  baselines  the  configura¬ 
tion  of  the  design/cquipment.  An  engineering  change  order  may  be  called  a  “Notice  of 
Revision,"  “Engineering  Change  Notice,”  “Revision  Directive,"  or  other  similar  terms 
according  to  local  practice.  The  designer  will  be  required  to  contribute  technical 
information  within  his  purview  to  this  document. 

Specification  Change  Notices  (SCNs) 

Whenever  a  change  to  a  design  or  equipment  is  authorized,  the  pertinent 
specification  and  engineering  drawings  must  be  changed  accordingly.  Instructions  for 
document  revision  should  be  contained  in  engineering  change  orders.  An  approved 
SCN  sheet  should  be  included  with  these  instructions  along  with  the  detailed  infor¬ 
mation  needed  to  produce  revised  copy  for  the  specification.  The  SCN  sheet  and 
revised  pages  prepared  for  the  specification  eventually  are  attached  to  the  latest 
approved  version  of  the  specification,  thus  updating  the  specification  to  agree  with 
the  updated  configuration  of  its  item.  Military  program-peculiar  specifications  must 
be  changed  only  by  using  DD  Form  1696  as  prescribed  by  MIL-STD-490.  Changes  to 
specifications  not  under  military  control  may  be  controlled  by  other  SCN  practices 
peculiar  to  the  individual  developer  or  manufacturer.  In  any  event,  the  designer  may 
be  called  upon  to  prepare  or  review  revised  portions  of  specifications  as  required  by 
SCNs  (which  themselves  are  usually  prepared  by  configuration  management  person¬ 
nel). 

Drawing  Change  Notices  (DCNs) 

DCNs  are  similar  to  SCNs  except  they  are  prepared  by  design  engineers  and 
implemented  in  accordance  with  DoD-STD-100. 

SUPPORT  DOCUMENTATION 

While  the  engineering  data  and  design  change  documentation  delineated  above 
are  primarily  applicable  to  the  design  and  development  cycles,  the  design  engineer 
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may  be  called  upon  to  provide  engineering  data  and  information  required  for  the 
preparation  of  such  support  documentation  as  technical  manuals  for  operation  and 
maintenance,  maintenance  standards  books,  computer  programming  documentation, 
operating  instruction  charts,  and  Held  change  documents. 

It  is  important  for  support  documentation  to  be  consistent  with  the  product 
and  the  product  documentation. 

Technical  Manuals 

The  design  engineer  generally  is  not  required  to  prepare  such  documentation 
himself,  as  technical  manual  writing  is  a  specialty  occupation,  but  he  may  be  required 
to  develop  the  source  data  for  the  manuals  and  is  in  the  best  position  to  review  and 
approve  manuscripts  for  completeness,  accuracy,  and  adequacy  of  purpose.  Commer¬ 
cial  manuals  purchased  with  commercial  off-the-shelf  items  should  be  required  to 
meet  the  minimum  standards  for  such  documentation,  as  delineated  in  MIL-M-7298. 
Manuals  for  systems  and  equipment  developed  by  or  for  the  military  services  must  be 
prepared  in  accordance  with  applicable  military  specifications  and  standards.  Each 
service  has  its  own  requirements  reflected  by  its  own  “limited”  specifications.  The 
military  specification  most  commonly  used  for  Navy  electronic  equipment  manuals  is 
MIL-M- 15071.  Technical  manuals  should  be  procured  at  NOSC,  through  NOSC  Code 
96  (Technical  Information  Division). 

Maintenance  Standards  Books 

These  documents  provide  equipment  maintenance  instructions  to  Navy  per¬ 
sonnel  in  the  field  when  the  Programmed  Maintenance  System  is  not  in  effect.  While 
these  documents  can  be  prepared  manually  from  data  already  in  the  associated  tech¬ 
nical  manuals,  engineers  may  be  required  to  provide  additional  information  to  the 
technical  writers.  Maintenance  standards  books  are  prepared  in  accordance  with 
MIL-B-21741. 

Operating  Instruction  Charts 

Usually  prepared  by  technical  writers  from  information  available  in  technical 
manuals,  these  charts  may  on  occasion  require  some  input  from  an  engineer. 

Field  Change  Documentation 

If  equipment  already  in  the  field  is  to  be  modified,  the  personnel  responsible 
for  making  such  modifications  require  clear,  complete  instructions  as  to  the  purpose 
and  method  of  modification,  checkout  after  modification,  and  other  factors.  In  addi¬ 
tion,  existing  technical  manuals  and  other  support  documentation  in  the  field  will 
require  changes  that  reflect  the  modification.  Engineers,  whether  or  not  they  were 
involved  in  the  initial  design  of  the  subject  equipment,  may  have  to  provide  to  the 
technical  writer  all  the  information  required  by  the  specification  to  which  the  field 
change  and  technical  manual  change  documentation  is  prepared.  The  specification  for 
field  changes  is  M3U-F-17655. 
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Computer  Programming  Documentation 

When  the  design  of  a  computer  or  any  associated  system  is  accomplished,  cer¬ 
tain  types  of  “user”  documents  are  required  to  ensure  that  operating  and  mainte¬ 
nance  personnel  have  the  required  information  for  the  performance  of  their  mission. 
(These  “user”  documents  are  in  addition  to  the  programming  specifications  that  must 
be  prepared  during  the  design  cycle.)  While  responsibility  for  providing  the  data  and 
information  for  user  computer  program  documentation  usually  falls  primarily  on  the 
programmer,  the  character  of  the  user  documents  is  such  that  both  the  programmer 
and  the  hardware  design  engineer  have  to  contribute  to  the  inputs  furnished  the  tech¬ 
nical  writer.  They  should  also  review  such  documents  for  completeness,  accuracy,  and 
adequacy.  Depending  upon  the  type  and  application  of  the  computer-associated  equip¬ 
ment  in  question,  there  are  several  specifications  or  instructions  covering  the 
requirements  for  content  and  format  of  the  various  documents  required.  MIL- 
STD-2167A  provides  guidelines  for  software  documentation  and  can  be  tailored  to 
peculiar  program  requirements. 

Parts  Provisioning  Documentation 

The  engineer  should  be  aware  of  the  requirements  for  provisioning.  Provision¬ 
ing  documentation  is  a  direct  result  of  maintainability  tasks  which  determine  the 
type  and  level  of  maintenance  actions  required  for  a  design.  The  engineer’s  direct 
input  to  maintainability  tasks,  therefore,  indirectly  afTocts  provisioning  actions  and 
associated  costs.  Provisioning  documentation  is  generally  prepared  in  accordance 
with  MIL-STD-15G1  by  a  logistician  or  by  a  technical  writer  who  specializes  in  this 
field. 

Other  Support  Documentation 

The  ongineor  must  sometimes  provide  data,  information,  and  guidanco  for  the 
preparation  of  documents  other  than  those  described  in  provious  subsections.  Refer¬ 
ence  standards  books,  operators’  handbooks,  maintenance  manuals,  and  training 
guides  are  examples  of  these,  along  with  other  specialized  documents  required  for 
training,  installation,  operation,  and  maintenance  in  the  field.  Support  documenta¬ 
tion  is  prepared  mainly  by  technical  writing  specialists,  howovor;  the  ongineor  is 
expected  to  provide  pertinent  information,  and  in  some  cases  to  contribute  written 
material. 

Project  Management  Support  Documentation 

Plans  in  considerable  variety  are  normally  requirod  to  support  the  project 
management  function.  Those  plans  should  be  kept  to  a  minimum;  however,  they  are 
sometimes  important  chocks  on  contractor  procedures: 

Configuration  Management  Plan 

Configuration  Status  Accounting  reports 

ILS  Plan 

Design-to-Cost  Data 
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Reliability  Program  Plan 
Maintainability  Program  Plan 
Environmental  Test  Plan/Procedures 
Human  Engineering  Program  Plan 
Maintenance  Engineering  Analysis  Program  Plan 
LOR  Program  Plan 
QA  Program  Plan 
Inspection  System  Program  Plan 
Production  Plan 
Accession  List 

The  use  of  an  accession  list  is  highly  recommended.  An  accession  list  is  a  peri¬ 
odic  report  of  all  the  documents  which  a  contractor  produces  for  internal  use;  the 
government  obtains,  through  the  accession  list,  access  to  this  information  for  only 
the  cost  of  reproduction.  The  data  required  by  the  government  in  the  contract  are 
limited  to  only  known  essontial  data  items,  thus  reducing  costs  by  not  asking  for  data 
which  may  or  may  not  be  nooded.  The  Statement  of  Work  must  contain  the  following 
task; 


“The  contractor  shall  prepare  and  update  (monthly)  a  list  of  all  reports, 
records,  memoranda,  plans,  procedures,  and  other  data  items  generated 
in  support  of  this  contract.  Upon  request  from  tho  Government,  the 
contractor  shall  provide  copies  of  specific  data  items.  The  Government 
shall  reimburse  the  contractor  for  reproduction  costs  incurred  in  this 
task.” 

The  CDRL  must  reforenco  the  task  in  tho  SOW.  Tho  CDRL  item  for  an  Accession  List 
Data/Internal  Data  can  cite  DIPs  DI-A-3027A  or  UD1-A-2G48G. 

An  accession  list  may  also  be  used  by  the  knowledgeable  systems  engineer  to 
watch  for  omissions  in  contractual  efforts.  When  documentation  should  be  created 
and  it  is  not,  the  systems  engineor  can  ask  appropriate  questions  and  redirect  efforts 
to  cover  the  omissions. 
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XXI.  RISK  MANAGEMENT 


The  project  manager  is  charged  with  the  responsibility  to  manage  the  tasks, 
funds,  schedules,  and  risks.'  Risks  are  encountered  by  everybody,  everyday,  in  every¬ 
thing  they  do,  from  simply  getting  out  of  bed  to  taking  a  trip  to  the  moon.  Most  risks 
are  so  small  or  inconsequential  that  most  everyone  ignores  them;  even  the  sometimes 
painful  consequences  are  shaken  off  as  bad  luck.  In  a  classic  example,  one  seldom 
considers  the  risk  of  pain  when  driving  a  nail  to  hang  a  picture,  and  a  throbbing  fin¬ 
ger  easily  obscures  the  concept  that  a  risk  was  taken.  A  complex  technical  project 
deserves  better  attention  to  risk  than  driving  a  nail,  and  recognizing  that  any  project 
ontails  many  risks  highlights  the  need  for  risk  management. 

Risk  management  has  been  developed  into  a  highly  effective  tool  by  most  suc¬ 
cessful  project  managers,  although  its  application  is  often  intuitive  However,  the 
insurance  industry  has  consciously  developed  risk  management  into  a  rigorous  sci¬ 
ence.  The  decisions  facing  a  technical  project  manager  will  seldom  yield  to  the  statis¬ 
tical  analysis  of  an  actuary,  but  the  qualitative  techniques  which  have  evolved  are 
very  useful.  The  sections  which  follow  discuss  risk  management  as  it  can  be  applied 
to  technical  projects.  If  this  guidance  at  least  stimulates  conscious  consideration  of 
risk  with  the  other  project  factors,  it  will  have  served  its  purpose. 


GENERAL  SUMMARY  OF  RISK  CONSIDERATIONS 


Risk  is  a  probability  of  fa  lure  to  achieve  a  well-defined  goal.  If  success  is  tho 
unqualified  achievement  of  the  goal,  the  probability  of  success  is  one  minus  the  risk, 
since  all  probabilities  are  measured  on  a  scale  from  zero  (no  chance  of  occurrence)  to 
one  (certainty  of  occurrence).  Success  and  failure  hinge  on  the  definition  of  the  goal, 
the  woll-definod  goal  consists  of  tho  following  characteristics: 

A  complete  description  of  that  to  be  attained  including  tolerances  on  each 
descriptive  parameter 

The  resources  to  be  spent  in  attainment 

The  time  in  which  the  goal  is  to  be  reached 

Any  constraints  which  may  exist  an  tho  actions  that  may  be  taken  to  achieve 
the  goal 

Obviously,  there  are  many  examples  of  partial  success  which  were  termed  oither  sat¬ 
isfactory  or  unsatisfactory;  this  directly  implies  two  underlying  risk  principles: 

1.  There  are  a  number  of  elements  which,  taken  together,  describe  the  goal; 
there  is  a  risk  associated  with  each  element. 


'DoD  Directive  6000.1  of  13  July  1971. 
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2.  There  are  consciences  attached  to  the  failure  to  attain  each  element 
which  may  vary  from  insignificant  to  catastrophic. 

These  seemingly  trivial  definitions  and  principles  form  the  foundations  of  risk  man¬ 
agement. 

Since  risk  and  the  probability  of  success  sum  to  one,  it  is  easy  to  jump  to  the 
conclusion  that  the  objective  of  risk  management  is  to  minimize  risk,  thereby  maxi¬ 
mizing  probability  of  success.  If  all  risk  could  be  reduced  to  zero,  success  would  be 
certain.  However,  it  would  seem  that  both  omniscience  and  omnipotence  would  be 
necessary  to  bring  this  condition  about.  More  practically,  some  finite  risk  must 
always  be  assumed  to  exist.  If  success  is  not  certain,  the  possible  consequences  of  fail¬ 
ure  must  be  considered.  This  adds  a  new  dimension  to  risk  management  —  that  of 
maximizing  the  probability  of  an  acceptable  outcome;  this  may  even  mean  a  lower 
probability  of  success  per  se. 

This  leads  to  the  problem  of  defining  what  constitutes  an  acceptable  and  an 
unacceptable  consequence.  For  purposes  of  this  discussion,  the  categories  will  be 
defined  as  follows: 


Critical 


Potentially  critical 


Probably  noncritical 


Noncritical 


A  failure  which  is  of  itself  unacceptable,  i.e., 
catastrophic 

A  failure  which  is  acceptable  but  which  is 
unacceptable  in  combination  with  other  failures 

A  failure  which  is  acceptable  and  must  occur  in 
combination  with  several  other  failures  to  lead  to 
an  unacceptable  result 

A  totally  inconsequential  failure 


Another  practical  problem  concerns  the  fact  that  eveiyday  goals  are  simply 
not  well  defined.  Indoed,  some  aspects  may  defy  practical  determination.  In  order  to 
develop  criteria  for  success,  to  identify  risks,  and  to  determine  the  consequences  of 
failure,  it  is  usually  necessary  to  estimate  aspects  of  the  missing  pieces.  The  estima¬ 
tion  process  at  its  best  relies  on  educated  guesses  and  at  its  worst  on  the  other  kind. 
In  each  case,  the  probability  of  estimating  error  and  its  consequences  must  be  consid¬ 
ered  in  tho  risk  assessment. 


The  four  characteristics  formulating  the  well-defined  goal  are  closely  interre¬ 
lated;  thus,  the  associated  risk  elements  are  interrelated,  Such  functional  relation¬ 
ships  will  obviously  affect  the  tactics  to  be  employed  in  managing  the  risks.  These 
relationships  may  be  categorized  in  the  following  manner: 

Independent  Not  affected  by  other  risk  elements 

Dependent  Totally  determined  by  the  outcome  of  other 

elements 


Interdependent  Functionally  related  to  other  elements  such  that 

each  affects  the  others 
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The  nature  of  the  goal  determines  the  exact  relationships  as  well  as  the  consequences 
associated  with  its  risks.  Nevertheless,  the  courses  of  action  for  dealing  with  the  risks 
must  clearly  tackle  independent  elements  directly,  attack  interdependent  elements  as 
a  group,  and  approach  dependent  elements  indirectly  by  manipulating  the  elements 
on  which  they  are  dependent. 

If  each  of  the  four  goal  characteristics  is  a  single  element,  it  is  relatively  sim¬ 
ple  to  analyze  the  goal,  to  assess  risks,  and  to  assign  consequences  to  those  risks; 
therefore,  the  risks  should  be  fairly  simple  to  manage.  Unfortunately,  this  is  not  nec¬ 
essarily  the  case.  Each  goal  element  is  toleranced;  i.e.,  a  range  of  acceptability  is 
definod  about  the  goal.  If  the  tolerance  is  tight,  it  will  be  more  difficult  to  stay  within 
the  range  than  if  the  tolerance  is  loose.  When  the  elements  are  interrelated  so  that 
one  must  be  traded  off  against  another,  tight  tolerances  make  this  task  harder.  If  the 
intolerance  is  sufficiently  loose,  tradeoffs  might  be  made  such  that  the  probable  con¬ 
sequence  of  the  associated  is  affected.  This  will  depend  on  the  nature  of  the  risk,  its 
relationships  with  other  risks  and  the  range  of  acceptability.  Because  the  manipulat¬ 
ing  of  one  risk  to  change  another  appears  intuitively  to  be  a  potent  tool,  it  seems  wise 
to  identify  thoso  risks  which  will  and  will  not  lend  themselves  to  alteration  by  this 
method;  accordingly,  the  risks  should  be  classed  as  follows: 


Rigid 

Those  elements  for  which  the  risk  consequences 
cannot  be  altered  regardless  of  tho  degree  of  risk 
assessed  or  of  relationships  with  other  elements 

Flexible 

Those  elements  for  which  the  risk  consequences 
can  be  altered 

A  risk  which  is  critical  (or  potentially  critical)  but  flexible  is  obviously  a  good  man¬ 
agement  target.  Conversely,  a  rigid,  noncritical  risk  makes  an  excellent  tradeoff 
against  related  targets. 

The  more  complex  the  goal  is,  the  more  important  it  is  to  be  able  to  categorize 
goal  elements.  Within  each  goal  characteristic,  many  subdivisions  are  normally  possi¬ 
ble  before  the  goal  is  partitioned  all  the  way  down  to  the  individual  element.  While 
the  subdivisions  may  vary  with  each  field  of  endeavor,  the  keys  to  discovering  the 
functional  relationships  between  the  elements  will  probably  be  disclosed  by  studying 
the  rules  of  partitioning.  Nonarbitrary  rules  of  partitioning  must  be  invertible;  other¬ 
wise,  there  could  be  no  way  to  synthesize  the  whole  from  the  parts.  The  inverted 
rules  (rules  of  integration)  will  describe  the  desired  relationships  by  necessity.  Since 
these  relationships  are  required  knowledge  for  the  risk  manager,  the  implication  is 
that  the  risk  manager  must  be  conversant  with  that  field  of  endeavor.  Very  complex 
goals  spanning  several  fields  will  require  a  multidisciplinary  background  for  the  man¬ 
agement  team,  embodied  in  the  management  leader. 

The  preceding  discussions  have  assumed  that  risk  management  can  be  reduced 
to  practice.  While  this  may  have  been  a  brash  assumption  for  the  general  case,  it  is 
heartening  to  note  that  an  entire  industry  is  built  on  risk  management  —  the  insur¬ 
ance  industry.  Given  the  apparent  success  in  that  field,  it  seoms  reasonable  to  assume 
that  somewhat  effective  techniques  must  be  available  to  manage  risks  in  a  cognitive 
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maimer  in  general.  In  reviewing  the  general  discussions  above,  there  would  seem  to 
be  at  least  eight  categories  of  technique: 

Defining  the  goal 

Identifying  risks 

Assessing  consequences 

Assessing  degree  of  risk 

Manipulating  consequences 

Manipulating  degree  of  risk 

Assessing  results 

Developing  a  best  plan  of  attack  to  achieve  a  goal 

Both  qualitative  and  quantitative  methods  are  needed  in  order  to  adequately  address 
the  life  cycle  of  attaining  a  goal  from  concept  to  success.  Such  results  should  describe 
practical  guidelines  of  great  utility  to  the  technical  project  manager. 

RISK  IDENTIFICATION 


The  first  step  in  risk  management  must  be  the  identification  of  the  risks  to  bo 
managed.  Indeed,  once  a  risk  is  identified,  its  management  often  becomes  obvious. 
There  is  no  cookbook  mothod  or  formula  to  identifying  risks;  at  any  givon  timo,  it  is 
nearly  impossible  to  identify  all  the  risks  for  a  project.  However,  the  following  proce 
dure  may  be  easily  adapted  to  fit  the  peculiarities  of  the  project  and  aid  in  risk  identi¬ 
fication. 

For  identification  purposes,  risks  may  be  considered  as  either  project  oriented 
or  technology  oriented.  Project-oriented  risks  are  those  which  are  associated  with  the 
goals  of  the  specific  project,  such  as  cost  targets,  schedules,  and  resource  constraints. 
Technology-oriented  risks  are  those  which  would  bo  essentially  the  same  regardless  of 
tlio  project  goals,  such  risks  are  usually  associated  with  achieving  a  specified  level  of 
performance.  There  are  also  political  risks  which  take  on  tho  character  of  cither  pro¬ 
ject-  or  technology-oriented  risks.  Opposition  to  project  budget  appropriations  will 
have  a  projoct-oriented  character,  for  instance,  opposition  to  the  operational  require¬ 
ment  or  technology  animosity  (as  against  nuclear  power)  may  bo  regardod  os  techni¬ 
cally  oriented. 

The  identification  of  projoct-oriented  risks  is  directly  facilitated  by  project 
planning.  The  better  a  project  is  organized,  the  easier  and  more  certain  the  risk  iden¬ 
tification  process  can  be.  By  utilizing  the  work  breakdown  structure  (WBS)  organ  nu 
tion  described  in  chapter  III,  Program  Planning,  a  risk  can  be  envisioned  for  the  per 
formance,  cost,  schedule,  and  availability  of  resources,  for  each  individual  task  ele¬ 
ment.  In  practice,  the  risk  associated  with  many  elements  will  be  so  small  as  to  be 
neglected.  However,  before  throwing  out  a  risk,  consider  its  impact  on  the  next 
higher  level  in  the  WBS,  if  the  impact  is  inconsequential,  the  risk  may  be  neglected. 
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Generally,  it  will  not  be  practical  to  consider  the  tasks  below  level  4  in  the  WBS,  but 
even  the  smallest  project  will  usually  be  concerned  with  tasks  at  the  third  level.  It  is 
handy  to  reproduce  the  project  WBS  and  to  circle  the  elements  which  present  the 
greatest  risk.  On  a  complex  project,  a  color  code  will  be  useful  to  identify  each  risk  as 
high,  medium,  or  low  and  the  risk  consequences  as  critical,  semicritical,  or  noncriti- 
cal.  Assume  that  opposition  will  develop  against  the  required  resources  for  a  task  to 
determine  whether  political  risks  exist.  Usually,  such  risks  will  impact  on  funding, 
but  organizational  relationships  may  also  present  risks  in  accessing  certain  facilities 
or  personnel  talent. 

The  identification  of  technology-oriented  risks  is  controlled  by  the  knowledge 
of  that  technology  available  to  the  project.  The  more  information  that  is  assimilated 
by  the  project,  the  easier  it  is  to  identify  and  manage  the  technical  risks.  The  infor¬ 
mation  may  como  from  several  sources: 

Knowledgeable  people 

Research 

Past  experience 

In  general,  the  gathering  of  information  requires  both  time  and  money.  If  a  constant 
effort  is  expended  to  gather  information,  the  information  base  will  tend  to  grow  as  a 
logarithmic  function  of  time.  For  these  two  reasons,  the  information-gathering  effort 
is  most  effective  at  tho  start  of  a  project.  To  limit  the  scope  of  effort  noedod,  it  is 
important  to  utilize  talented  personnel  who  have  experience  in  the  technology  in  the 
project  planning  phase;  they  will  know  much  of  the  required  information  and  will  also 
know  to  a  large  what  information  is  unknown.  Thus,  it  will  be  possible  to  organize 
rosearch  efforts  to  gain  that  information  which  is  needed  most.  Thero  will  always  be 
unanswered  questions  and  unknowns  to  which  the  project  will  be  naive,  so  some  risk 
must  always  bo  incurred.  Chapter  IV,  Conceptual  Phase,  contains  guidelines  for  gath¬ 
ering  information  and  for  organizing  the  technical  effort  in  accordance  with  the  per¬ 
ceived  risk.  The  fixod  political  risks  must  be  discovered  by  knowing  the  project’s 
“chain  of  command”  and  tho  attitudes  of  personnel  therein  either  directly  or  indi¬ 
rectly  and  also  by  being  familiar  with  tho  policies,  directives,  and  instructions  which 
will  apply.  Anyone  with  review  authority,  approval  authority,  or  budgeting  authority 
in  the  project’s  chain  of  command  can  pose  a  problem  or  a  solution;  it  is  important  to 
koep  in  touch  with  their  attitudes  to  discover  which  may  pose  risks  to  the  project. 
Once  the  project  is  underway,  valuable  experience  bocomes  available  to  identify  and 
manage  risks;  it  is  essential  to  document  this  experience  through  engineering  note¬ 
books,  drawings,  progress  reports,  and  memoranda,  especially  if  personnel  are  con¬ 
stantly  changing,  so  that  this  information  is  not  lost. 

The  risks  associated  with  estimating  may  be  considered  technology-oriented 
risks  even  though  they  may  concern  project  parameters  such  as  cost  and  schedule. 
Knowledgeable  people  and  their  past  experience  may  be  pivotal  in  making  accurate 
estimates.  Also,  they  will  be  essential  in  characterizing  the  accuracy  of  an  estimate 
from  any  source.  The  progress  toward  achieving  all  estimated  parameters  should  be 
tracked  through  a  testing  program  (for  technical  parameters)  or  an  appropriate  man¬ 
agement  information  system  (for  project  parameters).  A  lack  of  progress  can  be  used 
as  a  measure  of  risk. 
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RISK  CHARACTERIZATION  AND  ASSESSMENT 


When  a  nonnegiigible  risk  is  identified,  a  method  of  managing  it  must  be  for¬ 
mulated;  this  metb  4  will  depend  on  the  kind  of  risk  and  how  badly  it  can  affect  the 
project.  Risk  elements  may  be  high  or  low,  critical  or  noncritical,  flexible  or  rigid, 
dependent  or  independent,  and  so  forth  as  described  in  the  introduction  to  this  chap¬ 
ter. 


The  degree  of  risk  may  be  considered  inversely  proportional  to  the  knowledge 
of  the  associated  task  available  to  the  project.  One  may  assign  subjective  probabilities 
of  success  or  even  develop  statistical  probabilities  in  some  instances.  However,  it  is 
normally  sufficient  to  characterize  the  degree  of  risk  as  follows: 


Low 

The  task  has  been  done  before  and  this  experience 
is  available  to  the  project. 

Medium 

The  major  portions  of  the  task  have  teen  done 
before  and  the  experience  is  available  to  the 
project. 

High 

One  or  more  rntjor  portions  of  the  task  have  not 
been  done  before  or  the  experience  is  not 
available  to  the  project. 

The  principle  relating  knowledge  of  a  task  to  the  risk  is  well  established.  However,  if 
the  risk  is  puroly  a  matter  of  chance,  as  in  a  dice  throw,  the  statistical  probabilities 
of  success/failure  should  be  used.  When  an  estimated  parameter  —  cost,  schedulo, 
reliability,  or  whatever  —  is  measured  in  the  course  of  the  project,  a  measure  of  risk 
may  be  developed  by  looking  at  the  ratio  of  the  percent  of  goal  attainod  to  the  percent 
of  resources  remaining  to  accomplish  the  goal.  If  the  ratio  is  below  a  threshold,  say 
0.95,  consider  the  risk  low,  and,  conversely,  if  it  is  above  ceiling,  say  1.05  or  1.10, 
consider  the  risk  high.  For  the  primary  resource  (usually  funds),  this  ratio  will  be  the 
actual  expenditure  to  expected  expenditure  for  the  corresponding  degree  of  progress. 

The  project  WBS  can  reveal  how  critical  or  potentially  critical  or  noncritical 
and  how  dependent  or  independent  or  interdependent  the  risk  elements  are.  Tools 
such  as  the  PERT/critical  path  method  can  also  help.  No  specific  guidance  can  be  pro¬ 
vided  here  beyond  stressing  the  importance  of  honestly  evaluating  the  risk  elements 
as  they  affect  the  higher  levels  of  the  WBS  and  the  critical  path  of  the  project.  Each 
project  presents  its  own  difficulties  in  analyzing  task  relationships.  Often  it  will  be 
possible  to  develop  numerical  relationships  by  using  subjective  evaluations;  these 
relationships  are  particularly  useful  in  determining  where  to  expend  risk  manage¬ 
ment  resources. 

Risk  elements  can  almost  always  be  characterized  as  flexible.  Rigidity  is  nor¬ 
mally  imposed  by  limits  on  resources.  Since  flexibility  and  manageability  are  directly 
related,  it  is  important  to  identify  those  limits  and  to  plan  tasks  to  maintain  flexibil¬ 
ity. 
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HANDLING  RISK 


The  handling  of  risk  is  probably  the  most  effective  way  to  decide  bow 
to  allocate  project  resources;  after  all,  risk  management  strives  to  maximize 
the  chances  of  achieving  acceptable  results.  The  risk  decisions  can  be  used  to 
determine  not  only  how  resources  should  be  allocated  but  also  how  the 
resources  can  best  be  applied. 

Priorities  must  b8  established.  Tbe  risks  which  bear  the  most  critical 
consequences  should  be  handled  first.  Of  those  critical  risks,  high  risks 
should  be  treated  as  top  priority.  On  a  complex  project,  it  is  useful  to  assign 
priorities  to  the  risks  and  to  notate  the  WBS  accordingly.  Table  XXI-1  illus¬ 
trates  one  such  priority  system. 


Tabic  XX-1.  Risk  priorities. 


Probability  of  Failure 


Risk  Consequence 

High 

Medium 

Low 

Critical 

1 

2 

3 

Potentially  critical 

4 

5 

6 

Probably  noncritical 

7 

8 

9 

Noncritical 

10 

The  overall  project  risk  can  be  considered  acceptable  if  all  priority  1  through  5  risks 
can  be  managed.  When  tbe  risks  are  not  acceptable,  the  project  should  be  terminated. 


Next,  appropriate  techniques  must  be  selected  to  handle  the  risks.  Table  XXI-2 
lists  some  strategies  and  examples  for  handling  risks.  When  a  qualitative  method  has 
been  selected,  an  attempt  should  be  made  to  quantitatively  express  the  risk  situation 
as  it  applies  to  the  project.  This  effort  can  reveal  subtle  interactions  between  project 
risks  and  can  increase  confidence  in  decisions  based  on  the  method  even  though  sub¬ 
jectively  derived  probabilities  are  used.  A  risk  approach  can  be  modeled  for  virtually 
all  mqjor  project  decisions  including  source  selections.  The  extent  to  which  this  form 
of  decision  analysis  is  implemented  should  depend  on  the  scope  of  the  project.  In 
many  cases,  these  guidelines  will  have  served  their  purpose  if  the  project  manager 
consciously  considers  risks  along  with  the  technical,  cost,  and  schedule  factors  of  the 
project. 
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Table  Xxi-2.  Risk  strategies. 


Method  Application  Conditions 

I.  Hedging  Strategies:  Hedging  strategies  offset  risks  by  providing  alternatives 
which  have  low  risk  and  acceptable  results  although  the  ideal  goal  is  only 
partially  achieved. 

A.  Parallel-path  method  High,  technology-oriented  Two  or  more  paths  are 

risks  pursued  concurrently; 

the  paths  must  differ  in 
risk  characteristic  and 
at  least  one  path  must 
be  at  low  risk.  (Parallel 
paths  to  secure  price 
competition  do  not 
apply.) 

B.  Backup  method  Medium  and  low,  tech-  One  or  more  backup 

nology-oriented  risks  paths  are  identified; 

the  paths  must  differ  in 
the  area  of  risk.  For 
medium  risks, 
resources  to  implement 
the  backup  are  identi¬ 
fied  and  retained 

II.  Transfer  or  Exchange  Strategies:  Transfer  strategies  exchange  an  unknown  risk 
for  a  known  acceptable  risk  or  a  known  critical  risk  for  a  known  less-critical  risk. 

A.  Warranties  and  Unknown  project-oriented  A  known  price  (war- 

risk  ranty  cost)  is  paid  to 

fix  a  future  condition 
(usually  support  costs); 
the  warranty  cost  is 
then  budgeted.  The 
unknown  risk  is  sup¬ 
planted  by  a  funding 
risk. 


B.  Insurance  method  Critical  or  potentially  A  known  price  is  paid 
critical  project-oriented  to  limit  possible  future 

risks  consequences  as  in  fire 

insurance,  liability 
insurance,  etc.  This 
method  can  only  be 
applied  where  an 
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Table  XX1-2.  Risk  strategies  (continued). 


Method 

Application 

Conditicas 

B  (continued). 

“insurer”  can  be  found. 
Often  this  condition 
can  be  satisfied  con¬ 
tractually  with  a  sup¬ 
plier.  The  price  paid 
must  be  within  reason 
to  the  condition 
insured  against. 

C.  Contractual  method 

Same  as  (A)  and  (B) 

When  a  contract  is 
involved,  various  con¬ 
tractual  agreements 
can  be  made  to  transfer 
risks  to  the  contractor. 
The  contract  must  be 
fixed-price  and  nega¬ 
tive  price  incentives 
may  be  indicated. 

III.  Reduction  Strategies.  Reduction  strategies  amount  to  better  management  and  to 
careful  control  of  resources  which  lead  to  a  higher  probability  of  success.  They 
may  be  applied  to  virtually  any  risk  situation  including  in  combination  with 
other  strategies.  To  reduce  technical  risk,  top  personnel  would  be  assigned  to  the 
task.  Cost  and  schedule  risks  would  require  more  intensive  preplanning  and  must 
be  taken.  In  the  application  of  reduction  strategies,  care  must  be  taken  not  to 
create  risks  on  other  tasks  by  stripping  them  of  resources  and  not  to  interfere 
with  the  task  through  “micromanagement.”  If  problems  do  occur,  the  goal  is  to 
minimize  their  effect. 

IV.  Avoidance  Strategies.  Avoidance  strategies  are  alternate  paths  which  avoid 
exposure  to  the  particular  risk.  Primarily,  avoidance  strategies  are  applied  to 
technology-oriented  risks.  Examples  include  the  choice  of  one  technical  approach 
over  another  because  of  known  past  difficulties  with  the  latter.  A  pseudo 
technology-oriented  political  risk  might  be  avoided  by  structuring  the  project  for 
sponsorship  by  nonhostile  offices  or  to  bypass  a  hostile  office  in  the  approval 
cycle.  The  crux  of  any  avoidance  strategy  is  knowing  that  a  risk  exists  and 
devising  a  means  of  nonexposure. 

V.  Evasion  Strategies.  Evasion  strategies  utilize  prior  planning  to  avoid  risk  con¬ 
sequences  even  though  a  failure  occurs.  Because  of  legal  and  technical  restraints, 
evasion  strategies  are  almost  wholly  limited  to  political  risks.  Usually  this 
requires  developing  a  “power  base”  of  influentii  “friends  of  the  project”; 
occasionally  the  “snow  job”  is  a  viable  tool.  The  ultimate  success  of  an  evasion 
strategy  in  a  technical  project  application  relies  on  honest  facts  and  the  apparent 
ultimate  achievement  of  the  goai. 
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Table  XXI-2.  Risk  strategies  (continued). 


Method  Application  Conditions 

VI  Risk  Assumption:  Assumption  strategies  recognize  the  unlikelihood  of  the 
expected  Applied  to  project-oriented  risks,  especially  funding  and  scheduling, 
assumption  strategies  plan  “padding”  into  project  estimates;  the  key  to  successful 
risk  assumption  is  limiting  the  extent  of  the  padding  to  what  is  sufficient  while 
ensuring  an  adequate  amount  Applied  to  technology-oriented  risks,  assumption 
strategies  tend  toward  ultraconservative  design  parameters.  Care  must  be  taken 
not  to  misapply  such  strategies,  especially  to  technical  problems,  since  they  tend 
to  produce  gross  overspecification  with  drastically  increased  costs.  Risk  assump¬ 
tion  must  not  become  a  “cover  your  anatomy”  ploy. 


BETTER  ESTIMATES  USING  RISK  ASSESSMENTS 


An  estimate  is  a  judgment  as  to  what  is  most  likely,  that  is,  it  is  an  expected 
value  When  estimates  are  made  at  a  low  level  in  the  WBS,  the  combined  estimate  at 
the  top  level  is  more  accurate  because  there  is  less  chance  of  omitting  key  elements 
and  because  there  is  a  statistical  tendency  toward  eliminating  the  collective  error  in 
the  estimates  However,  a  simple  combination  of  estimates  yields  the  most  expected 
value  for  the  estimated  parameter  disregarding  the  effects  of  interdependent  and  de¬ 
pendent  tasks  Because  of  task  dependencies,  an  overrun  in  a  task  can  be  propagated 
to  other  tasks  whereas  an  underrun  can  remain  isolated,  therefore,  a  “best  estimate’ 
will  almost  always  be  overrun.  A  subjective  risk  assessment  technique  can  be  applied 
to  correct  higher-level  estimates  to  yield  a  much  more  precise  estimate 

As  an  example,  let  us  assume  that  a  work  package  consists  of  n  tasks  for 
which  a  parameter  is  being  estimated  in  an  aniuunt  n;;  The  parameter  may  be  cost, 
time  to  completion,  or  any  other  expended  resource.  Figure  XXI-1  shows  the  tradi¬ 
tional  estimate  for  the  work  package. 


*2  *3  »4  *n 

Figure  XXI-1  The  traditional  estimate 


Generate  a  probability  of  success  for  each  task  and  designate  it  Pn;  although  the 
probability  is  subjective,  the  best  estimate  of  the  knowledgeable  estimator  of  an  will 
be  a  sufficiently  accurate  Pa  There  is  now  an  identified  risk  of  1-Pn  Should  a  failure 
occur,  additional  resource  will  be  utilized  to  attempt  to  complete  the  task;  this  effort 
will  also  have  a  probability  of  failure/success.  This  additional  effort  is  a  conditional 
task  n’  Figure  XXI-2  illustrates  the  combined  work  package  estimate. 
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Any  number  of  reiterations  may  be  accounted  for  until  a  sufficiently  high  probal 
of  success  is  reached.  In  the  illustration,  task  3  has  a  second  conditional  task  n" 


which  deponds  upon  the  failure  of  the  first  conditional  task  n';  on  the  other  hand, 
task  4  is  sufficiently  certain  that  no  condition  task  is  estimated.  The  technique  is  sur¬ 
prisingly  easy  to  use  and  becomes  easier  with  experience.  In  conjunction  with  stan¬ 
dard  estimating  techniques,  this  method  produces  estimates  which  are  approximately 
an  order  of  magnitude  moro  accurate  for  complex  projects  If  the  project  is  using  a 
computerized  WBS,  this  method  can  be  integrated  into  the  program. 


The  risk  assessment  estimate  will  be  higher  than  the  traditional  estimate. 
Allocate  the  estimated  resources  to  tasks  according  to  the  traditional  method  (which 
represents  the  most  likely  or  expected  amount)  and  bold  the  excess  in  a  management 
reserve.  This  management  reserve  is  justified  by  the  estimated  risk. 
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CHAPTER  XXII.  PROJECT  MASTER  PLAN 
(PMP) 


Project  planning  and  control  can  be  done  without  writing  everything 
down— but  it  is  much  better  to  commit  project  plans  to  paper.  It  is  not 
the  paper  itself  that  is  so  important;  it  is  the  process  of  arriving  at  the 
detailed  information  that  goes  into  the  paper  and  the  process  of  review¬ 
ing  and  updating  the  paper  that  is  important.  A  Project  Master  Plan 
encompasses  all  aspects  of  the  program  plan,  system  engineering,  and 
acquisition  strategy.  It  serves  as  a  “contract”  between  the  project  man¬ 
ager,  the  systems  engineer,  and  the  task  leaders.  It  serves  as  catalyst  to 
bring  together  the  project  resources,  the  project  goals,  the  system  de¬ 
sign/implementation,  the  acquisition  environment  (the  rules  and  regu¬ 
lations  which  apply  to  the  project),  and  the  project  environment  (the 
organizational  relationships  and  task  responsibilities  and  management 
tools). 

This  section  does  not  dictate  inviolable  rule,  rather,  it  is  hoped  that 
this  section  will  be  found  to  have  invaluable  guidance. 

DESIGNING  A  PROJECT  MASTER  PLAN 


In  designing  a  PMP,  it  is  useful  to  keep  several  principles  in  mind. 

e  The  planning  process  is  as  important  as  the  plan  itself. 

•  The  PMP  should  be  a  living  document  that  changes  as  project  require¬ 
ments  and  environments  change  and  as  more  detail  becomes  available. 

•  The  PMP  is  a  useful  “contract”  within  the  project  management  organiza¬ 
tion  only  if  the  project  manager,  the  systems  engineer,  and  the  principle 
task  leaders  participate  in  mutually  planning  the  tasks.  A  PMP  is  not  an 
end  in  itself  and  should  not  be  generated  just  to  prove  that  lots  of  money 
can  be  spent  cutting  down  trees  and  putting  ink  on  pulp. 

•  A  plan  is  only  good  if  it  is  implemented  correctly. 

•  A  plan  documents  intended  future  actions  rather  than  past  performance. 
While  it  may  acknowledge  past  decisions,  its  value  is  in  the  decision/action 
roadmap  that  serves  to  connect  the  present  to  the  desired  goals.  When  cir¬ 
cumstances  dictate  a  detour  in  the  plan,  the  plan  must  change  to  account 
for  the  “now”  rather  than  for  the  “expected”  when  the  plan  was  first  for¬ 
mulated. 

•  A  good  plan  anticipates  the  possible  detours  and  variations  in  future  cir¬ 
cumstances  and  provides  alternate  routes  to  the  project  goals  This  func¬ 
tion  is  an  important  element  of  managing  risks. 

A  PMP  should  be  generated  by  the  project  manager,  the  systems  engineer,  and  the 
key  task  leaders— the  project  management  team.  It  may  not  be  possible  to  get  all  of 
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the  required  task  leaders  together  at  the  same  time;  this  is  acceptable  as  long  as  task 
leaders  for  tasks  which  interface  with  each  other  can  participate  in  the  planning 
process  together.  If  one  or  more  significant  tasks  is  being  accomplished  under  con¬ 
tract,  the  task  leader  will  not  be  known  until  after  the  contract  is  let,  so  use  the 
person  who  will  be  responsible  for  monitoring  the  contract.  The  task  leaders  should 
be  selected  to  be  knowledgeable  in  the  critical  issues  associated  with  their  tasks  — 
outstanding  in  their  field.  Occasionally,  a  personality  conflict  will  exist  between  task 
leaders  whose  tasks  must  interface;  it  then  becomes  incumbant  upon  the  systems 
engineer  or  project  manager  to  arrange  “shuttle  diplomacy”  or  other  means  to  ensure 
that  effective  communications  do  take  place.  Often  the  communications  modes  estab¬ 
lished  between  task  leaders  during  the  planning  process  become  the  modes  which  are 
used  during  the  execution  of  the  project,  so  it  is  important  for  the  systems  engineer/ 
deputy  project  manager  to  monitor  the  planning  process  and  to  ensure  that  effective 
interpersonal  relations  and  communications  are  established. 

The  PMP  design  process  should  generally  follow  the  steps  below.  A  rigorous 
adherence  to  these  steps  is  not  required,  but  the  thought  processes  they  represent  and 
the  project  team  relationships  they  entail  should  be  present  in  the  planning  process. 
Virtually  every  successful  project  studied  has  followed  this  general  planning  pattern 

1.  Identify  the  project  goals  and  the  acquisition  environment.  This  normally 
involves  detailed  discussions  with  the  project  sponsorship  and  may  involve 
requirements  definition  tasking. 

2.  Select  project  leaders.  The  project  leaders  can  be  participants  in  the 
requirements  definition  process,  key  people  in  the  functional  organization 
having  the  project  management  responsibilities,  and  experts  from  key  sup¬ 
porting  organizations. 

3.  Enroll  the  project  leaders  in  the  project  goals.  Each  participant  must  have 
a  clear  concept  of  what  is  to  be  accomplished  and  be  able  to  project  what 
role  their  task  plays  in  achieving  those  goals.  The  enrollment  process  may 
be  as  simple  as  a  detailed  briefing  and  review  of  project  documentation  to 
visits  to  prospective  user  organizations  and  participation  in  exercises 
which  illustrate  the  problems  to  be  resolved  by  the  project  product.  The 
extensiveness  of  the  enrollment  process  may  be  tailored  to  the  needs  of  the 
prospective  task  leader. 

4.  Stratcgize.  A  strategy  establishes  what  kinds  of  tasks  need  to  be  accom¬ 
plished  and  the  mtqor  milestones  to  be  achieved  in  ordor  to  attain  the  pro¬ 
ject  goals.  All  of  the  project  leadership  should  be  involved  in  developing  the 
strategy. 

5.  Identify  the  task  interfaces. 

G.  Estimate  the  resources  required  by  each  task  (costs,  time,  manpower,  tal¬ 
ent,  facilities,  equipments,  contracts,  and  risks).  Task  leaders  on  interfac¬ 
ing  tasks  should  work  with  each  other  to  develop  these  high-level 
estimates.  These  estimates  will  be  top-down  in  detail  and  from  time- 
present  to  timo-complete  in  flow  (forward-looking  estimates). 

7.  Breakdown  the  long-term  milestones  and  mqjor  tasks  into  intermediato- 
and  short-term  milestones  and  more  detailed  taks.  Continue  this  break- 
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down  process  as  long  as  it  is  supportable  by  available  information. 
Together  with  the  acquisition  environment,  use  this  information  to  formu¬ 
late  a  Work  Breakdown  Structure  (WBS)  and  an  appropriate  schedule 
tracking  system  (GANTT  or  PERT,  etc.). 

8.  Identify  the  information  gaps  and  the  approaches  needed  to  fill  these  gaps. 
This  may  cause  additional  tasks  to  be  identified. 

9.  Develop  detailed  estimates  for  each  detailed  task  for  all  near-term  mile¬ 
stones  (all  the  identified  milestones  which  have  no  predecessor  milestones 
from  the  present  planning  time)  and  all  tasks  which  are  dependent  on 
these  immediate  tasks.  (The  use  of  the  risk  estimating  technique  in  chap¬ 
ter  XXI  is  recommended.) 

10.  Develop  estimates  for  other  known  tasks. 

11.  Accumulate  estimates  from  the  bottom  to  the  top  of  the  WBS  and  from 
time-complete  to  the  present  (working-back  estimates). 

12.  Reconcile  the  top-down  estimates  with  the  bottom-up  estimates,  and  the 
forward-looking  estimates  with  the  working-back  estimates.  The  reconcili¬ 
ation  process  will  identify  risks  caused  by  inadequate  resources  (especially 
time  and  funding)  and  potential  alternatives  which  might  be  incorporated 
in  the  overall  strategy. 

13  Document  the  plan.  Actually,  the  documentation  should  be  generated  as 
the  planning  process  progresses,  hut  it  is  useful  to  incorporate  the  docu¬ 
mentation  into  a  formal  document  which  can  be  referenced  and  which  can 
serve  as  a  baseline  for  future  revisions. 

14.  Repeat  steps  4  through  13  every  time  a  significant  milestone  is  achieved. 

15.  Repeat  steps  2  through  6  whenever  a  key  task  leader  is  replaced. 

There  are  many  schemes  for  actually  reducing  these  steps  to  practice.  The  best  PMP 
design  tactics  take  the  project  team  personalities  into  account.  There  are  some 
schemes  which  have  a  proven  track  record  of  failure,  the  most  prevalent  of  these  fail¬ 
ure  modes  include  the  following  (1)  when  the  project  manager  or  the  project  sponsor 
dictates  all  elements  of  the  planning,  (2)  when  a  “belt-way  bandit”  (support  contrac¬ 
tor  not  actually  involved  in  the  work)  writes  the  plan,  or  (3)  when  the  PMP  is  gener¬ 
ated  to  fulfill  the  need  of  having  a  document  rather  than  as  a  planning  instrument. 
The  successful  schemes  provoke  thoughtful  planning  of  both  strategies  and  tactics  for 
achieving  the  project  goals  and  promote  communications  between  the  members  of  the 
project  management  team. 


USES  OF  A  PMP 


The  primary  purpose  for  creating  a  PMP  is  as  an  instrument  of  monitoring 
and  controlling  the  project.  The  PMP  design  process  becomes  a  method  of  enrolling 
the  project  management  leadership  into  a  team  and  of  promoting  communications 
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between  task  leaders.  The  project  leadership  must  have  a  basis  for  mutual  trust  and 
confidence.  The  task  leaders  must  have  a  clear  understanding  of  their  responsibilities 
authorities,  and  accountability.  It  is  understood  that  the  many  gray  areas  between 
tasks  cannot  be  unilaterally  resolved  by  a  single  task  leader;  gray  areas  must  be  re¬ 
solved  by: 

mutual  consent 

authority  delegated  by  the  project  manager 

higher  authority  (project  manager  or  systems  engineer) 

Usually,  the  project  manager  does  not  know  where  all  the  gray  areas  are,  so  the  task 
leaders  must  be  trusted  to  recognize  which  problems  must  be  resolved  by  which 
method  The  task  leaders  are  prepared  to  do  this  only  if  they  share  the  project  man¬ 
agement  goals  A  clear  concent  of  one’s  personal  responsibilities/authorities.does  not 
require  a  clearly  defined  task  in  a  management  TEAM 

A  well-developed  PMP  serves  as  a  contract  between  the  project  manager  and 
the  other  team/task  leaders.  The  viability  of  this  contract  is  totally  dependent  on  the 
personal  commitment  of  each  individual.  The  project  manager  seldom  has  formal  per¬ 
sonnel  authority  over  the  members  of  the  team,  so  the  discipline,  either  encourage¬ 
ment  or  punishment,  that  can  be  imposed  is  limited.  It  may  be  limited  to  an 
encouraging  word,  a  favorable  memo  to  the  supervisor,  an  occasional  certificate  of 
appreciation,  or  a  chocolate  doughnut,  or  it  may  be  limited  to  a  private  “chowing  out" 
or  request  to  the  supervisor  for  action.  This  extent  of  discipline  is  sufficient  when  the 
individuals  have  personal  commitment. 

The  PMP,  or  et  least  an  executive  summary  of  the  PMP,  can  be  used  as  a  con¬ 
tract  between  the  project  sponsor  and  the  project  management.  Seldom  can  a  sponsor 
deliver  all  of  the  funding  when  it  is  ideally  required.  Likewise,  requirements  change, 
unscheduled  briefings  are  needed,  schedules  of  ships  and  facilities  needed  for  project 
tasks  change,  and  other  elements  change.  The  PMP  serves  as  a  baseline  for  managing 
these  changes.  Some  of  these  changes  can  be  made  “within  scope,"  but  others  will 
require  additional  funding  or  a  modification  of  tasking.  The  PMP  serves  the  impor 
tant  role  of  identifying  the  impact  of  changes  and  helps  the  project  manager  to  avoid 
getting  backed  into  a  losing  corner.  Good  sponsors  appreciate  a  PMP  because  the  risk 
consequences  of  their  decisions  can  be  determined  and  because  adequate  justifications 
are  easier  to  develop  for  project  resources  and  project  defenses  (a  project  must  be 
defended  somewhere  in  the  project’s  political  environment  about  once  a  week),  other¬ 
wise,  the  sponsor  is  left  to  make  decisions  in  the  blind  which  may  adversely  alter  the 
acquisition  environment  in  an  untimely  way.  In  ot^er  words,  the  PMP  is  useful  to 
the  project  and  to  the  project's  chain  of  command. 

An  executive  summary  of  the  PMP  can  be  useful  in  obtaining  a  form  of  a  pro¬ 
ject  charter  A  project  charter  formally  establishes  the  project’s  authorities  and  work¬ 
ing  relationships  within  a  functional  organization.  The  projoct  manager  can 
sometimes  get  his  or  her  personal  administrative  chain  of  command  to  review  and 
“approve”  the  PMP  and  then  successfully  negotiate  special  delegations  of  authority 
which  can  make  the  project  more  efficient.  The  project  manager  can  also  use  the  PMP 
as  a  basis  of  negotiating  working  relat.onships  with  the  various  supporting  organiza¬ 
tions,  agreements  with  these  organizations  can  be  formalized  in  Work  Agreements 
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signed  by  the  supervisors  in  the  support  organizations  and  the  head  of  the  project’s 
line  organization.  Besides  highlighting  special  project  requirements  of  which  the  sup¬ 
porting  line  organizations  should  be  aware,  the  Work  Agreement  helps  to  enlist  the 
commitment  of  the  supporting  line  organization  beyond  the  personal  commitment  of 
its  project  participants.  The  commitment  of  line  management  is  essential  to  obtaining 
the  support  of  the  greater  organization  in  getting  the  resources  needed  to  do  work 
(supply,  accounting,  special  test  facilities,  shops  and  labs,  space,  screen  rooms,  and  so 
forth).  An  actual  project  charter  is  rarely  issued,  but  if  the  participating  line  organi¬ 
zations  can  be  informally  committed  to  the  project  goals,  the  charter  is  not  necessary. 

PMP  STRUCTURE 


A  PMP  should  be  modular,  because  not  all  elements  will  be  produced  at  the 
same  time  or  by  the  same  organization.  Also,  the  PMP  is  a  living  document  and  sub¬ 
ject  to  change.  The  following  structural  elements,  outlined  below,  are  suggested. 
These  elements  are  neither  exhaustive  nor  mandatory;  nevertheless,  they  offer  a 
flavor  for  what  should  be  accomplished  by  a  PMP.  It  is  suggested  that  a  specification- 
style  numbering  system  be  employed  to  enable  ready  reference  to  particular  informa¬ 
tion  of  intorest  through  the  table  of  contents.  The  PMP  should  be  a  daily  working 
tool,  so  it  should  be  kept  unclassified,  if  at  all  possible.  Aspects  which  are  classified 
may  be  published  as  a  separate  classified  addendum  or  included  by  reference  to  for¬ 
mally  published  documents  (TRs,  TDs). 

ABSTRACT  —  an  abstract  is  suggested  but  not  mandatory.  The  abstract 
should  contain  one  or  two  paragraphs  describing  the  project  goals  in  “sales  brochure” 
prose. 

EXECUTIVE  SUMMARY  —  the  executive  summary  should  contain  a  sum 
mary  of  tho  project  background,  a  summary  of  the  operational  requirements,  and  the 
project  charter. 

LIST  OF  EFFECTIVE  PAGES  —  this  is  almost  mandatory  because  a  PMP  is 
likely  to  be  very  dynamic  if  it  is  actually  being  used.  All  the  PMP  users  need  to  know 
that  their  working  copy  of  tho  PMP  is  up  to  date.  It  is  suggested  that  the  page  num¬ 
bers  be  numbered  in  sequence  within  each  major  section  (1-1,  1-2,  2-1,  etc.) 

TABLE  OF  CONTENTS  —  an  indentured  table  of  contents  is  suggested.  (An 
indentured  table  of  contents  indents  headings  which  are  subordinate  to  other  head¬ 
ings.  Major  headings  are  left  justified  while  sections  are  indented  once,  subsections 
are  indented  twice,  etc.) 

1.  INTRODUCTION 

1.1  BACKGROUND 

1.1.1  HISTORY  —  events  leading  to  the  establishment  of  the 

project. 

1. 1.2  SUMMARY  OF  PRIOR  RELATED  EFFORTS 

1.1.3  REFERENCES  —  especially  official  documents  such  as  the 
OR  and  TEMP  plus  significant  reports  and  project  documents  referenced  in  the  PMP 
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1.2  OPERATIONAL  REQUIREMENTS— This  should  include  all  of  the 
elements  found  in  the  OR  (OPNAVINST  5000.42)  but  not  limited  by  the  page  limita¬ 
tions  of  the  OR.  Subordinate  paragraphs  may  expand  significantly  on  the  known 
operational  requirements  and  may  prioritize  requirements.  A  prioritization  of 
requirements  is  particularly  important  when  all  requirements  cannot  be  addressed 
within  the  scope  of  the  currently  funded  efforts.  Also,  this  section  should  distinguish 
between  required  thresholds  of  performance  and  desired  thresholds.  When  system 
requirements  tradeoff  analyses  are  completed,  they  may  be  included  in  this  section  or 
referenced,  if  they  are  formally  published. 

1.3  CONCEPT  OF  EMPLOYMENT  —  This  may  be  included  here  in  its 
entirety  or  included  by  reference  to  a  formally  published  document.  If  published  sepa¬ 
rately,  an  unclassified  abstract  is  desireable  in  this  section.  Any  operational  inter¬ 
faces  between  the  project  system  product  and  other  operational  systems  should  be 
included.  Concept  of  employment  scenarios  should  be  sufficiently  complete  to  illus¬ 
trate  operator  interactions  with  the  system. 

1.4  CONCEPT  OF  SUPPORT  —  This  should  include  support  require¬ 
ments  peculiar  to  the  concept  of  employment  scenarios  and  the  overall  support  strat¬ 
egy  References  to  LSA,  LOR,  and  ILS  documentation  should  be  included  when  it  is 
generated. 


1.5  PROJECT  CONSTRAINTS  AND  THRESHOLDS  -  Costs,  sched¬ 
ule,  and  performance  elements  should  each  be  identified.  The  performance  thresholds 
should  include  combat  operational  performance,  peace-time  operational  performance 
(if  different  from  combat),  and  support  element/systems  effectiveness  performance 
They  should  cross-reference  to  the  OR  or  TEMP  as  appropriate. 

1.6  SPECIAL  ACQUISITION  CONSIDERATIONS  -  Other  projects 
which  are  related  by  operations  or  technology  should  be  noted  here.  Also,  any  special 
restrictions  which  may  affect  the  acquisition  strategy  should  be  documented  in  this 
subsection. 


1.7  RISK  PERCEPTION  SUMMARY-The  external  factors  which 
affect  the  overall  risk  of  the  project  should  be  noted  here.  These  might  include  an 
overly  ambitious  schedule,  inadequate  funding,  a  highly  dynamic  threat,  dependency 
on  immature  technologies,  or  inadequately  defined  requirements.  Attention  should  be 
drawn  to  those  sections  of  the  PMP  which  are  specifically  structured  to  address  these 
risks  by  referencing  the  PMP  appropriate  paragraphs. 

1.8  SECURITY  CLASSIFICATION  GUIDE  -  If  a  separate  guide  is 
available,  it  may  be  referenced  here.  Generally  it  is  necessary  to  identify  project- 
peculiar  security  issues  and  to  establish  the  classification  policies  implementing  the 
more  general  requirements  of  higher  authority.  Authorities  for  classification  are  pro¬ 
vided  here  The  Security  Officer  can  provide  much  of  the  detail  required  for  tliis  sub¬ 
section. 

2.  PROJECT  ORGANIZATION  —  The  first  3  subsections  are  often  simply 
lists  players,  including  the  organization,  the  designated  point-of-  contact,  the  tele¬ 
phone  number,  facsimile  number,  e-mail  address,  and  mailing  address  for  each  entry. 
A  back-up  point-of-contact  is  also  desireable. 
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2.1  SPONSOR  PARTICIPANTS  —  Those  organizations  and  points- 
of-contact  which  form  the  project  chain-of-command  (and  which  obtain  project  fund¬ 
ing  and  determine  mqjor  milestones  and  set  operational  performance  thresholds). 

2.2  DESIGN  PARTICIPANTS  —  Those  key  persons  and  organization 
under  the  control  or  authority  of  the  project  manager  for  execution  of  project  tasks. 

2.3  SUPPORT  PARTICIPANTS  —  Those  organizations  and  individuals 
who  are  tasked  to  support  the  project  manager  but  who  do  not  come  under  the  project 
manager’s  authority.  (Typically  this  includes  operational  units  which  are  tasked  by 
higher  authorities  to  provide  operational  expertise,  to  participate  in  user  conferences 
or  advisory  panels,  to  provide  test  facilities,  and  the  like.) 

2.4  ORGANIZATIONAL  RELATIONSHIPS,  FUNCTIONS,  AND 
RESPONSIBILITIES  —  This  subsection  spells  out  who  does  what  to  whom  on  the 
project  A  subsubsection  may  spell  out  those  authorities  reserved  by  the  sponsor.  The 
project  charter  is  normally  included  under  the  project  manager’s  paragraphs.  The 
taskings  to  the  various  supporting  organizations  is  normally  also  included  here. 

2.5  PROJECT  ORGANIZATIONAL  STRUCTURE  -  This  is  normally 
provided  in  chart  form  showing  lines  of  authority  and  special  lines  of  communication. 
Points  of  contact  normally  are  repeated  here  with  their  telephone  number. 

2.6  PROJECT  FACILITIES  —  Special  facility  requirements  and  the 
organizational  responsibilities  for  providing  and  maintaining  those  facilities  are 
noted  in  this  subsection.  Of  special  interest  are  requirements  for  secure  facilities  and 
controlled  access  facilities  and  for  the  control  of  COMSEC  material.  The  project  pro¬ 
cedures  associated  with  secure  facilities  should  be  reviewed  by  the  Security  Officer. 
The  project  procedures  associated  with  COMSEC  materials  must  be  reviewed  by  the 
COMSEC  Material  Custodian 

2.7  PROJECT  DOCUMENTATION  CONTROL  -  It  may  be  necessary 
to  establish  a  project  library.  Consult  with  the  NOSC  Librarian  to  determine  what 
procedures  are  appropriate  to  maintain  and  control  a  document  library.  If  classified 
material  is  involved,  the  procedures  should  be  cleared  through  the  Security  Officer.  If 
intelligence  information  is  involved,  tho  Intelligence  Officer  should  be  consulted  to 
ensure  that  proper  procedures  are  followed.  In  any  case,  the  responsibility  for  main¬ 
taining  the  current  status  of  project-generated  documentation  should  be  clearly 
defined. 

3.  PROJECT  STRATEGY  —  This  section  summarizes  the  mqjor  tasks  and 
milestones  and  the  management  tools  necessary  to  accomplish  the  project.  It  also  pro¬ 
vides  the  justifications  and  rationales  for  the  major  procurements  anticipated  for  the 
project.  Some  of  the  detail  for  this  section  will  not  become  available  until  later  in  the 
project,  but  the  nuyor  eloments  will  be  known  at  the  beginning.  This  section  should 
be  updated  whenever  the  TEMP  is  updated. 

3.1  MAJOR  TASKS  AND  MILESTONES -This  subsection  should  be 
very  similar  to  the  TEMP  in  content.  However,  this  section  should  also  contain  a 
rationale  for  each  task  and  milestone,  explaining  how  each  task  fits  into  the  overall 
project  picture.  The  overall  costs  of  each  task  will  also  be  summarized  here.  It  is  sug¬ 
gested  that  a  summary  integrated  schedule  be  included  here  using  the  schedule  man¬ 
agement  tool  selected  for  the  pi  eject. 
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3.2  MANAGEMENT  CONTROLS  —  The  management  information  sys¬ 
tems  which  will  be  used  and  the  reporting  requirements  of  the  project  participants 
will  be  spelled  out  in  this  subsection.  Both  formal  and  informal  procedures  will  be 
covered;  the  rationale  for  the  program  assessment  methods  will  be  included.  The  pro¬ 
cedures  implementing  management/organizational  controls,  schedule  control,  cost 
controls,  technical  performance  measure  tracking,  and  risk  controls  will  be  included, 
as  required. 

3.3  MAJOR  PROCUREMENT  PLANS  —  Each  individual  contract 
anticipated  for  the  project  requires  advanced  planning;  this  subsection  provides  the 
rationale  and  justifications  for  each  contract.  For  contracts  which  are  large  enough  to 
require  an  externally  approved  ACQUISITION  PLAN  or  Advanced  Procurement 
Plan,  these  plans  will  be  appended  to  this  subsection.  This  section  does  not  cover 
minor  contracts  for  material  nor  small  purchases  of  project  parts,  but  it  does  include 
small  support  taskings  to  outside  organizations. 

3.4  RESOURCE  SUMMARY  —  The  required  funding  profile  will  be 
shown  here.  Also,  special  resources  will  be  identified  such  as  facilities,  test  resources, 
organizational  talent,  special  equipments,  and  the  like.  Each  resource  should  be  tied 
to  the  mqjor  tasks  or  milestone  with  which  it  is  associated.  As  a  minimum,  the  date 
of  required  availability  should  be  stated  in  the  same  level  of  detail  as  that  provided  in 
the  TEMP. 

4.  WORK  BREAKDOWN  STRUCTURE  (WBS)  -  The  WBS  should  be  pre¬ 
pared  in  accordance  with  MIL-STD-881  and  provided  in  this  section.  It  is  probably 
wise  to  maintain  a  three-level  executive  summary  WBS  for  distribution  outside  of  the 
project.  When  the  PMP  is  distributed  to  outside  organizations,  this  summary  WBS 
should  be  substituted  in  this  section.  Refer  to  HOW  TO  USE  A  WORK  BREAK¬ 
DOWN  STRUCTURE,  below. 

5.  SYSTEM  DESIGN  PLAN  —  This  section  provides  the  detailed  planning 
for  the  CONCEPTUAL  PHASE  and  the  DEMONSTRATION  AND  VALIDATION 
PHASE.  Each  plan  will  include  the  detailed  tasks,  deliverables,  schedules  and  mile¬ 
stones,  and  resource  summaries  for  the  phase.  The  rationales  fov  the  system  design 
decisions  will  be  incorporated  for  each  task  as  the  system  analyses  and  tradeoffs  ore 
performed;  this  is  normally  done  by  appending  tho  minutes  and  reporting  documents 
for  the  design  reviews  to  this  section.  The  task  leader,  performing  organization,  and 
special  organizational  requirements  will  bo  noted  for  each  task.  Even  when  projects 
do  not  have  a  formal  conceptual  phase  or  a  formal  validation  phase,  the  elements  of 
the  system  design  plan  should  be  developed  and  included  in  this  section.  Even  when 
the  project  is  proceeding  as  a  nondevelopmental  item  (NDI),  the  system  design  plan¬ 
ning  elements  will  be  documented  in  this  section.  When  tho  system  design  is  complete 
and  validated,  this  section  will  document  how  the  system  design  and  the  acquisition 
strategy  are  related. 

5.1  CONCEPTUAL  PHASE  PLAN  —  In  developing  and  reporting  the 
conceptual  phase  plan,  MIL-STD-1521  requirements  for  tho  SYSTEM  REQUIRE¬ 
MENTS  REVIEW  and  for  the  SYSTEM  DESIGN  REVIEW  should  bo  incorporated  in 
detail  as  appropriate  to  the  project.  All  plans  for  tho  execution  of  technical  investiga¬ 
tions,  breadboarding,  exploratory  development,  and  field  investigations  will  be 
included  in  this  subsection.  Upon  completion  of  this  phase,  the  system  specification 
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will  be  referenced  from  this  subsection.  The  system  specification  may  be  attached  to 
this  section  if  it  is  not  otherwise  subject  to  formal  document  publication  controls. 
Each  requirement  of  the  system  specification  will  be  referenced  to  an  operational 
requirement  or  to  a  system  tradeoff  report;  this  may  be  done  by  a  table  presented  in 
this  subsection. 

5.2  DEMONSTRATION  &  VALIDATION  (D&V)  PHASE  PLAN  -  In 
developing  and  reporting  the  D&V  phase  plan,  MIL-STD-1521  (and  MIL-STD-2167A 
for  software)  requirements  for  the  INTERFACE  DESIGN  REVIEWS  and  the  PRE¬ 
LIMINARY  DESIGN  REVIEW  (PDR)  should  be  incorporated.  All  plans  for  technical 
demonstrations,  brassboarding,  advanced  development,  and  operational  assists  should 
be  detailed  in  this  subsection.  The  method  of  validation  should  be  referenced  to  each 
system  specification  requirement  (analysis,  simulation,  or  testing)  and  the  results 
should  be  documented  and  referenced  in  this  section.  Upon  the  completion  of  this 
phase,  the  development  specifications  will  be  referenced  in  this  subsection,  and  each 
development  specification  requirement  will  be  referenced  to  either  the  system  specifi¬ 
cation  or  a  design  analysis  report,  this  may  be  done  by  a  table  presented  in  this  sub¬ 
section. 

6.  TRANSITION-TO-PRODUCTION  PLAN  -  This  section  provides  detailed 
plans  for  Full-Scale  Engineering  Development  (FSED)  (if  any),  for  TECHEVAL  and 
OPEVAL,  for  preproduction  prototyping,  and  for  the  implementation  of  the  product 
assurance  plans  in  the  production  phase.  The  plan  should  include  the  detailed  tasks, 
deliverables,  schedules  and  milestones,  and  resouce  summary.  The  task  leader,  per¬ 
forming  organization,  and  special  organizational  requirements  will  be  noted  for  each 
task.  Unless  formally  reported  elsewhere,  a  subsection  entitled  “configuration  con¬ 
trol'  should  be  incorporated  to  document  design  changes,  including  the  specification 
requirements  affected,  the  rationale  and  justification  for  approving  the  change,  an 
impact  assessment,  and  the  actions  taken  to  implement  the  change.  The  plan  should 
take  into  account  the  requirements  of  MIL-STD-1521  (and  MIL-STD-  21G7A  for  soft¬ 
ware)  for  the  Critical  Design  Review,  the  Test  Requirements  Review,  tho  Functional 
Configuration  Audit,  the  Physical  Configuration  Audit,  and  the  Formal  Qualification 
Review  The  production  or  procurement  specification(s)  requirements  should  be  refer¬ 
enced  to  the  system  specification,  the  development  specifications,  or  to  quality  provi¬ 
sions  derived  from  detailed  design  decisions,  this  may  be  done  in  a  chart  or  table.  The 
actual  contents  of  this  section  vary  considerably  from  projoct  to  project,  so  no 
detailed  guidance  can  be  provided  hero.  Projocts  developing  a  one-of-a-kind  system 
have  substantially  different  requirements  than  those  which  will  lead  to  large-scale 
production  In  goneral,  those  projects  with  small  production  requirements  will  pro¬ 
vide  more  actual  detail  in  this  section,  whereas  projects  having  large  production 
requirements  will  incorporate  information  into  the  PMP  by  reference  and  summary 
tables. 


7.  PRODUCTION/PROCUREMENT  PHASE  PLAN  -  This  section  provides 
the  detailed  plans  for  buying  the  necessary  number  of  systems.  The  details  will 
include  the  advanced  procurement  planning  necessary  for  the  production  contract(s), 
including  the  long-term  plans  for  multiple-souixe,  leader-follower,  warranty-weighted, 
and  other  types  of  production  contracts.  Plans  for  First  Article  Testing,  follow-on 
testing,  interim  training  and  logistics  support,  disposition  of  preproduction  assets, 
and  disposition  of  production  tooling  and  test  fixtures  are  all  to  be  included  in  this 
section,  as  appropriate. 
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8.  OPERATIONAL  SUPPORT  PHASE  PLAN  -  This  section  may  include 
plans  for  implementing  and  maintaining  logistics  support,  logistics  status  reviews, 
implementing  and  tasking  an  in-service  engineering  agent,  implementing  preplanned 
product  improvements  (PPPI),  and  other  tasks  determined  by  the  ILS/LSA  tasks  for 
the  in-service  system  support. 

9.  SYSTEM  PHASE-OUT  PLAN  —  This  section  incorporates  the  advanced 
planning  for  system  phase-out.  It  may  incorporate  performance  thresholds  for 
identifying  the  onset  of  system  obsolescence  (and  the  need  to  develop  a  replacement) 
or  cost  thresholds  derived  from  the  system  life-cycle  cost  model  which  might  indicate 
the  need  for  action  to  restore  or  replace  the  system.  If  the  system  contains  hazardous 
or  precious  materials,  this  section  can  be  used  to  cover  the  disposal  or  recovery  proce¬ 
dures  and  identify  the  organization(s)  responsible. 

10.  DETAILED  INTEGRATED  SCHEDULE  —  A  detailed  schedule  derived 
from  all  of  the  plans  (sections  5  through  9)  and  incorporating  all  of  the  identified 
work  packages  from  the  WBS  is  provided  in  this  section  using  the  selected  schedule 
management  tool  (see  3.2).  The  integrated  schedule  shows  task  dependencies  and 
interdependencies  and  when  information  is  expected  to  become  available  for  generat¬ 
ing  more  detailed  information  for  follow-on  phase  planning.  Ideally,  the  schedule  is 
calendar-based  and  includes  expected  achievement  dates,  earliest  expected  achieve¬ 
ment  dates,  and  latest  allowable  achievement  dates.  (An  achievement  date  is  either  a 
date  of  delivery  for  product  oriented  tasks  or  a  decision  date  for  project-oriented 
tasks.) 


11.  LIFE-CYCLE  COST  MODEL  —  The  project  LCC  model  is  an  important 
decision  tool;  this  section  documents  the  model  and  provides  the  underlying  assump¬ 
tions  and  rationales  for  nonstandard  elements  of  the  model.  The  actual  dotailed  LCC 
estimates  are  normally  not  included  in  this  section  but  may  be  attached  as  an  appen¬ 
dix.  It  is  suggested  that  the  LCC  model  elements  be  keyed  to  the  WBS. 

APPENDICES  —  PMP  appendices  shouid  include  management  planning 
documents  which  can  stand  alone  or  which  are  required  os  separate  documents  by 
outside  agencies  or  which  provide  supporting  detailed  information  to  sections  of  the 
PMP  (If  those  documents  are  separately  published  and  controlled,  a  single  appendix 
which  provides  e  management  executive  summary  for  each  can  be  included  instead  ) 
The  commonly  encountered  plans  include  the  following  topics' 

Advance  Procuroment/Acquisition 

Configuration  Management 

Data  Management 

Design-to-(unit  production)-Cost 

Electromagnetic  Compatibility/TEMPEST 

Environmental 

Facilities 

Human  Engineering 
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ILS 


Logistic  Support 

Maintainability 

Quality 

Packaging,  Handling,  Storage,  and  Transportation 
Personnel  and  Training  (and  Manpower) 

Product  Assurance 
Reliability 
Safety 
Security 

Standardization/Parts  Control 
System  Engineering  Management 

Testability  ^ 

T&E  ? 

Value  Engineering 

It  can  bo  seen  that  there  is  a  potential  for  lots  of  detailed  information,  result¬ 
ing  in  a  voluminous  document.  In  some  cases,  this  is  desireable  because  the  PMP 
becomes  the  central  source  of  information  for  project  decisions  and  becomes  the  key 
document  for  recording  the  project  history.  Small  projects  do  not  need  to  be  burdened 
with  substantial  publication  costs,  so  the  evolutionary  generation  and  maintenance  of 
a  PMP  can  be  a  cost-effective  solution.  On  the  other  hand,  large  projects  may  have  so 
many  details  to  manage  that  the  PMP  simply  serves  as  a  summary  and  roadmap  ref¬ 
erence  to  all  those  details  These  details  may  reside  in  a  host  of  supporting  documents 
that  may  include  a  System  Engineering  Management  Plan  (see  MIL-STD-499A)  and 
the  multitude  of  plans  supporting  systems  effectiveness,  product  assurance,  and  logis¬ 
tics  support  This  host  of  supporting  plans  may  ha'  s  a  lot  of  information  which  d'.pli- 
cates  parts  of  the  PMP,  often  word  for  word,  but  there  is  much  utility  in  a 
stand  alone  document  used  by  a  substantial  group  of  project  participants  who  share 
common  interests  and  who  need  not  be  burdened  by  details  outside  of  their  project 
tasking  The  result  is  that  a  good  PMP  for  a  large  project  may  be  substantially 
smaller  than  that  for  a  small  project. 

Although  there  is  a  multitude  of  considerations  in  the  management  and  svs 
terns. engineering  cf  a  project,  these  considerations  are  tailored  to  the  project  envirun 

mectsandreguLr  .m-nts.  so  the  conduct  of  the  project  results  in  a  system  acquisition 

that  is  both  i  flee*. ire  and  efficient  The  PMP  should  reflect  this  tailoring  process  that 
.s  incorporat'd  in  the  strategies  and  tactics  of  the  project  management  planning. 
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HOW  TO  USE  A  WORK  BREAKDOWN 
STRUCTURE  (WBS) 


OVERVIEW 

Work  breakdown  structures  have  long  been  used  for  project  planning,  estimat¬ 
ing,  and  control.  The  basic  structure  for  a  WBS  is  defined  in  MIL-STD-881.  However, 
merely  constructing  a  MIL-STD-881  WBS  will  not  provide  a  useful  tool  for  any  of 
these  project  uses.  In  fact,  the  WBS  is  used  in  markedly  different  ways  for  each  pur¬ 
pose.  A  complete  work  breakdown  structure  consists  of  three  main  sections: 

•  structural  breakdown 

•  work  packages 

•  estimating  categories 

These  sections  are  interrelated  to  provide  the  usefulness  of  tho  WBS  tool  and  to  en¬ 
able  interfacing  the  WBS  to  other  program/project  management  techniques  such  as 
Gantt  charts  and  project  flow/sequence  diagrams.  This  is  important  because  the  WBS 
provides  a  special  planning  visibility  to  the  project  that  other  techniques  do  not  pro¬ 
vide,  but  the  other  techniques  offer  perspectives  that  cannot  be  visualized  through 
the  WBS.  For  instance,  a  WBS  provides  a  detailed  relationship  of  tasks  and  subtasks 
which  is  essential  for  cost  estimating,  but  it  has  no  framework  for  displaying  schedul¬ 
ing  information.  Each  of  tho  WBS  sections  can  bo  structured  in  a  variety  of  ways; 
howevor,  it  is  recommended  that  the  guidance  provided  here  be  followed  in  order  to 
avoid  some  of  the  pitfalls  normally  encountered  and  to  derive  the  full  benefit  of  the 
WBS. 

STRUCTURAL  BREAKDOWN 

The  structural  breakdown  is  the  main  portion  of  a  work  breakdown  structure. 
Tho  structural  breakdown  starts  a*  the  very  fop  task  —  the  entire  project  —  and  pro¬ 
ceeds  to  divide  or  break  down  the  task  to  successively  more  detailed  subtasks.  Each 
breakdown  iteration  is  callod  a  level.  The  fop  lovel  (level  1)  is  always  the  total  project 
There  are  two  approaches  toward  this  breakdown  pi  ocess: 

•  task-oriented 

•  functionally  oriented 

The  task-oriented  approach  achieves  its  breakdown  through  logical  divisions  decided 
strictly  by  the  project/product  task  characteristics  without  regard  to  the  means  of 
accomplishment,  sequence  of  work,  or  team  assigned  to  the  task.  The  functionally  on- 
euted  approach  breaks  down  each  task  by  the  functional  project  team  assignments 
(i.e.,  code,  company,  or  organizational  component)  and  then  by  tasked  responsibili¬ 
ties. 


The  functionally  oriented  approach  is  used  to  support  various  forms  of  work 
agreements  across  organizational  lines.  It  is  strong  in  displaying  who  is  responsible 
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for  accomplishing  a  task,  but  it  is  weak  as  a  planning  or  estimating  tool.  Use  of  a 
functionally  oriented  WBS  often  results  in  a  failure  to  expose  all  of  the  tasks  to  be 
accomplished  and  in  large  underestimates.  This  normally  occurs  because  the  func¬ 
tional  breakdown  does  not  reflect  the  system  design  decisions  which  partition  the 
product  into  its  functional  components;  this  obscures  naturally  occuring  product 
interfaces  which  must  be  controlled.  However,  the  functional  breakdown  does  empha¬ 
size  organizational  interfaces  and  the  needs  for  project  communications. 

Task-oriented  approaches  are  assumed  by  MHrSTD-881.  The  task-oriented 
approach  is  recommended  because  it  provides  a  more  consistent  framework  for  all 
forms  of  estimating  and  because  it  usually  is  more  effective  in  helping  the  project 
planners  to  identify  obscure  or  hidden  tasks.  When  properly  instituted,  a  task- 
oriented  WBS  can  do  everything  achieved  by  a  functionally  oriented  WBS,  but  the 
opposite  is  not  true.  Except  for  parenthetical  comments,  the  remaining  text  will  only 
address  task-ori9nted  approaches  to  work  breakdown  structures. 

The  task-oriented  structure  consists  of  two  main  sections: 

•  product  tasks 

•  project  tasks 

Product  tasks  are  those  which  arc  directly  related  to  the  design  and  production  of  the 
product.  As  such,  the  product  tasks  describe  a  breakdown  of  the  miyor  assemblies, 
set,  groups,  and  other  units  which  make  up  the  product.  The  levels  recognized  by 
military  specifications  are  listed  in  table  XX1I-1.  Project  tasks  are  those  which  relate 
to  the  management,  installation,  testing,  documentation,  and  support  needed  to  de¬ 
velop,  implement,  and  support  tho  product.  MIL-STD-881  defines  the  project  tasks  for 
the  second  and  third  levels  of  the  WBS.  These  second-  and  third-level  project  tasks 
are  summarized  in  table  XX1I-2. 

Table  XXII-1.  Levels  of  product  comp'exity. 

Lffil  Description 

1  Sy  stem  —  a  composite  of  equipment,  skills,  and  techniques  capa¬ 
ble  of  performing  or  supporting  an  operational  role,  or  both.  A 
plote  system  includes  all  equipment,  related  facilities,  material, 
software,  services,  and  personnel  required  for  its  operation  and 
support  to  tho  degree  that  it  can  be  considered  a  self-sufficient 
unit  in  its  intended  operational  environment.  (M1L-STD-280A) 

2  Subsystem— a  combination  of  sets,  groups,  etc.,  which  performs  an 
operational  function  within  a  system  and  is  a  mqjor  subdivision  of 
the  system.  (Examples:  Data  processing  subsystem,  guidance  sub¬ 
system).  (MIL-STD-280A) 

3  Set  —  a  unit  or  units  and  necessary  assemblies,  subassemblies, 
and  parts  connected  together  or  used  in  association  to  perform  an 
operational  function.  (Examples:  Radio  receiving  set,  sound  mea¬ 
suring  set.  “Set”  is  also  used  to  denote  a  collection  of  related  items 
such  as  a  “tool  set”  or  “drawing  set”  or  “set  of  tires”.) 
(MIL-STD-280A) 
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Table  XX1I-X.  Levels  of  product  complexity  (continued). 

Level  Description 

4  Group  —  a  collection  of  units,  assemblies,  or  subassemblies 
which  is  capable  of  performing  a  complete  operational  function.  A 
group  may  be  a  subdivision  of  a  set  or  may  be  designed  to  be 
added  to  or  used  in  conjunction  with  a  set  to  extend  the  function 
or  utility  of  the  set.  (Example:  Antenna  group.)  (MIL-STD-280A) 

5  Unit  or  Weapon  Replaceable  Assembly  —  an  assembly  or  any 
combination  of  parts,  subassemblies,  and  assemblies  mixed  to¬ 
gether,  normally  capable  of  independent  operation  in  a  variety  of 
situations.  A  unit/WRA  possesses  specific  attributes  which  can¬ 
not  be  broken  into  constituent  parts  without  losing  those  attrib¬ 
utes.  These  attributes  are  usually  such  that  they  are  recognizable 
by  operators/field  personnel.  (Examples:  Radio  receiver,  internal 
combustion  engine,  hydraulic  jack.)  (MIL-STD-280A  and  MIL- 
STD-1390) 

6  Assembly  —  a  number  of  parts  or  subassemblies  or  any  combina¬ 
tion  thereof  joined  together  to  perform  a  specific  function  and 
capable  of  disassembly.  (MIL-STD-280A) 

7  Subassembly/Shop  Replaceable  Assembly  —  two  or  more  parts 
which  form  a  portion  of  an  assembly  or  a  unit  replaceable  as  a 
whole,  but  having  a  part  or  parts  which  are  individually  replace¬ 
able.  (M1L-STD-280A  and  MIL-STD-1390B) 

Note:  the  distinction  between  an  assembly  and  a  subassembly  is 
determined  by  the  individual  application.  An  assembly  in  one 
instance  may  be  a  subassembly  in  another  where  it  forms  a  portion 
of  an  assembly. 

8  Module  —  two  or  more  component  pieces  joined  together  which 
are  not  subject  to  disassembly  without  destruction  of  the 
designed  use.  (Note:  a  modulo  may  or  may  not  be  repairable;  if  it 
is  repairable,  it  is  repairable  at  the  depot  level.) 

9  Piece  Part  —  an  item  which  is  not  subject  to  disassembly  with¬ 
out  destruction  of  the  designed  use.  (Examples:  Composition 
resistor,  electron  tube,  audio  transformer.)  (MIL-STD-280A) 
(Note:  a  piece  part  is  never  repairable.) 


Work  packages  are  the  task  assignments  which  actually  accomplish  the  project 
work.  In  a  task-oriented  structure,  they  will  be  found  near  the  bottom  of  the  break¬ 
down  structure.  In  fact,  they  v/ould  always  be  the  lowest  level  except  that  a  WBS  may 
be  carried  to  greater  detail  for  estimating  purposes  than  is  sensible  for  task  assign¬ 
ment  purposes  (In  a  funtionally  oriented  structure,  the  work  packages  always  start 
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at  level  2,  and  subsequent  levels  are  known  as  work  details).  The  general  catagories 
of  work  packages  include  electronic  design,  software  design,  mechanical  design,  pro¬ 
totype  development,  fabrication,  item  test,  item  documentation,  item  procurement, 
etc.;  i.e.,  all  the  activities  necessary  to  develop,  produce,  and  support  the  item  repre¬ 
sented  at  the  next  highest  level  in  the  WBS. 

PROJECT  ORGANIZATION  AND  PRODUCT  TASK  BREAKDOWN 

At  the  beginning  of  a  system  development,  the  complexity  of  the  system  may 
not  be  definable  (until  some  design/development  work  is  accomplished),  so  the  work 
packages  will  appear  as  higher  level  tasks.  The  system  design/definition  process  is  a 
part  of  Systems  Engineering,  which  is  defined  as  a  project  task.  As  the  system 
becomes  defined,  the  work  packages  are  broken  down  into  component  tasks  and 
appear  at  the  more  detailed  levels;  the  system  units  defined  become  the  higher  level 
tasks  in  their  stead.  These  higher  level  tasks  will/should  follow  the  system-partition¬ 
ing  decisions  which  are  embodied  in  the  system  design  concepts. 

As  design  tasks  proceed  from  concepts  through  detailed  design  and  product 
design,  ever  greater  levels  of  detail  are  identified  down  to  the  minutest  of  compo¬ 
nents.  The  WBS  can  be  carried  down  to  this  highly  detailed  level  (and  might  be  in 
some  instances  for  estimating  purposes  or  to  track  risks);  however,  the  WBS  work 
packages  are  maintained  at  tho  level  identified  in  the  system  design.  (It  is  possible  to 
break  down  the  work  packages  to  these  detailed  levels,  but  there  is  little  or  no  useful 
information  gained  while  it  becomes  more  complex  to  track).  Work  packages  occur 
at/near  the  bottom  of  the  structural  breakdown  and  represent  tho  actual  task  assign¬ 
ments  to  performing  organizations  and  task  managers.  A  task  manager  should  be 
identified  for  each  work  package.  Although  a  single  individual  may  be  the  task  man¬ 
ager  over  several  WBS  work  packages,  several  guidelines  should  be  observed: 

•  the  work  package  tasks  should  be  similar  in  nature 

•  tho  task  manager  should  have  a  single  supervisor  in  the  project  organiza¬ 
tion 

•  the  task  manager  should  not  supervise  more  than  three  or  four  work  pack¬ 
ages  at  the  most 

If  these  guidelines  cannot  be  met,  the  WBS  work  packages  may  be  at  the  wrong  WBS 
level  (usually  too  detailed  level)  or  the  project  organization  may  be  flawed  (which  may 
simply  mean  that  it  would  be  wise  to  find  an  additional  task  manager). 

At  each  higher  task  level,  there  should  be  an  “integration  and  assembly  (1/A)” 
task  broken  out.  This  I/A  task  represents  the  actual  activity  of  putting  together  all  of 
the  lower-level  pieces.  The  I/A  work  package  assures  that  the  appropriate  estimates 
are  considered  when  accumulating  estimates  from  the  most  detailed  to  the  system 
level;  this  avoids  a  common  pitfall  of  doing  a  wonderful  job  of  estimating  the  details 
but  forgetting  their  ultimate  assembly  into  a  system.  (If  a  separate  I/A  task  is  not 
broken  out,  the  integration  and  assembly  estimates  are  added  to  the  accumulated 
estimate  at  the  next  highest  level;  thus,  the  accumulated  estimate  should  always 
exceed  the  sum  of  the  next  lowest  level  estimates  from  which  it  comes.  The  separate 
I/A  task  avoids  this  apparent  inconsistency.) 
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Work  packages  are  typically  the  only  activities  that  get  carried  over  into 
scheduling  information  systems  and  technical  flow/sequence  networks  from  the  WBS. 
It  is  in  these  other  techniques  that  task  dependencies  (sequencing)  are  displayed.  The 
common  element  used  to  hold  together  the  work  packages  when  displayed  in  these 
other  techniques  is  the  WBS  element  number. 

The  activities  more  detailed  than  a  work  package  are  usually  reflected  on  a 
drawing  tree  (summarizes  hardware  design  details),  a  flow  chart  (summarizes  soft¬ 
ware  design  details),  or  an  activity  listing  (summarizes  project  details  which  are  not 
reflected  in  the  design,  such  as  “procurement”  or  designer  participation  in  project 
tasks).  (A  project  organization  may  be  responsible  for  many  work  packages.  It  is  com¬ 
mon  for  the  organizational  manager  to  accumulate  the  activity  listings  for  all  work 
packages  within  the  group  into  a  functional  WBS.  This  allows  the  manager  to  easily 
visualize  the  work  assignments  and  to  ensure  the  organization  is  responsive  to  the 
needs  of  the  project.)  Activity  listings  are  simply  a  listing  of  all  the  various  actions 
which  are  necessary  to  accomplish  the  work  package.  These  may  include  “design  cir¬ 
cuits,”  “order  parts,"  “write  procurement  documents,”  and  other  common  and  repeti¬ 
tive  actions;  task  managers  often  maintain  activity  lists  in  the  form  of  a  Gantt  chart 
Usually,  activity  lists  and  drawing  trees  are  not  considered  part  of  the  WBS.  Activity 
lists  (usually  under  the  title  of  “subproject”)  are  considered  part  of  the  task  manager’s 
documentation,  and  drawing  trees  and  flow  charts  are  considered  part  of  the  engi¬ 
neering  documentation. 

The  system  design  documentation  should  track  exactly  with  the  WBS  because 
the  product  task  breakdown  should  be  derived  from  the  system  partitioning.  The  sys¬ 
tem  design  specification  tree  should  directly  reflect  the  WBS  and  vice  versa. 

WBS  ELEMENT  NUMBER 


The  clement  number  is  usually  constructed  to  carry  the  information  of  the 
WBS  structure  Each  level  of  the  WBS  is  numbered  to  show  the  higher  level  structure 
of  which  it  is  a  part  There  are  many  ways  to  formulate  this  element  number,  and 
there  is  no  particular  favored  method.  One  method  of  clement  numbering  is  described 
in  figure  XXII-1;  it  has  the  advantage  of  inherent  level  identification,  structure  en¬ 
coding,  and  functional  assignments  of  work  packages.  The  method  described  in  figure 
XXII-1  is  highly  recommended,  but  it  is  by  no  means  the  only  good  method  of  num¬ 
bering  a  WBS  In  choosing  a  numbering  system,  you  should  consider  the  following 
factors’ 

•  it  should  clearly  identify  WBS  levels 

•  element  dependencies  should  be  readily  distinquished 

•  it  should  be  compatible  with  the  other  information  systems  and  project 
management  tools  (some  computer  programs  severely  limit  the  field  size 
used  for  the  WBS  element  number) 

•  cost  center  factors  and  task  management  responsibilities  should  be  visible 
(i.e.,  the  project  organizational  chart  should  be  related  to  the  WBS  and 
this  relationship  displayed  in  the  numbering  scheme) 
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product  tasks,  project  tasks,  and  work  packages  should  be  distinquishable 
from  each  other  (if  not  accomplished  by  the  element  numbering  system, 
colors  can  be  useful) 


CH03.1.6.2.3.E 

CH03  -  project  number.  Using  the  project  number  or  some  similar  project  Identifier  for  the  top- 
level  element  number  serves  to  distinguish  the  WBS  from  others  and  maintains  a  means  of  read¬ 
ily  Identifying  the  WBS  level  by  observing  the  periods  separating  the  levels.  For  brevity,  a  general 
character  such  as  or  •*’  can  be  used,  but  be  careful  that  the  character  selected  doesn’t 
have  a  special  use  In  any  spreadsheet  or  automated  project  management  tool  you  are  using 

.f  6.2.3.  -  the  work  breakdown  of  project  and  product  levels.  ".1 "  Is  the  second  level,  *  6"  is 
the  third  level,  and  so  forth  for  as  many  levels  that  are  created  and  displayed  It  is  recommended 
that  *1*  through  ■9"  be  used  for  project  tasks  since  there  are  nine  project  tasks  specified  in  MIL- 
STD-88f  which  can  be  used  for  every  project  or  program  (MIL-STD-881  project  tasks  are  summa¬ 
rized  in  table  2).  For  product  tasks,  use  “10, '  “20,"  “30,"  .  or  letters  which  would  be  descriptive 
of  the  subsystem  (such  as  “C"  for  communications  subsystem  and  “L"  for  launch  subsystem),  but 
ensure  that  letters,  it  used  here,  differ  from  the  work  package  descriptors. 

.E  -  the  work  package  descriptor  The  letter  Is  used  to  be  readily  distinguishable  as  a  work  pack¬ 
age  tt  is  recommended  that  the  letters  be  selected  to  be  unique  to  a  task  manager  or  to  the  func¬ 
tional  pedotming  organization  performing  the  task  Also,  tt  is  useful  for  the  letter  to  be  related  to 
tho  nature  ol  the  work  package,  l.e  ,  “E“  for  electronic  design,  “M“  for  mechanical  design,  and  so 
forth  Letter  combinations  can  be  used  to  provide  more  Information  For  Instance,  “E9“  might  be 
used  to  show  electronic  design  performed  and  managed  by  Code  90  as  opposed  to  “E5"  for 
electronic  design  performed  by  Code  50  “Ej“  mlglit  denote  “an  electronic  design  task  managed 
by  John",  and  “Sk“  might  denote  “a  software  design  task  managed  by  Karen",  the  work  package 
descriptor  Is  consistent  throughout  the  WBS  so  that  sorts  can  easily  be  run  on  each  descriptor  to 
extract  information  for  tracking  progress,  managing  resources,  and  making  refined  estimates 


Figure  XXII-1 .  Recommended  WBS  element  numbering 

In  any  case,  the  element  number  is  the  “glue"  which  holds  together  both  the  WBS 
and  the  other  management  information  systems  used  by  the  project. 

USING  THE  WBS  AS  AN  ESTIMATING  FRAME 

The  WBS  is  tremendously  powerful  as  an  aid  to  estimating  and  tracking 
resources.  Typically,  the  resources  are  estimated  at  the  work  package  level  and  accu¬ 
mulated  to  the  top  of  the  WBS.  Thus,  if  the  work  package  is  level  7  in  the  WBS,  the 
estimates  are  made  at  level  7.  Level  C  will  reflect  all  of  the  estimates  at  level  7  com¬ 
bined  as  appropriate  to  describe  their  effect  at  level  6.  (Most  resources  are  additive, 
such  as  costs,  workhours,  etc.,  but  some  are  nonlinear,  such  as  risk  factors,  and  are 
combined  using  a  formula  If  unit  production  costs  are  being  estimated,  each  item  is 
accumulated  according  to  the  number  of  times  it  is  used  in  the  next  highest  level ) 
Each  estimating  category  has  its  own  rules  for  accumulating  the  estimate  to  the  next 
level  The  process  is  repeated  up  to  level  1,  so  that  the  resource  requirement/impact 
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is  visible  at  each  level.  The  typical  estimating  categories  encountered  in  DoD  are  as 
follows: 

bid  costs  (development  cost  goals) 
development  costs 

design-to-cost  targets  (unit  production  cost  goals) 

unit  production  costs 

support  cost  targets 

support  costs 

life-cycle  cost  targets 

life-cycle  costs 

cost  risks 

technical  performance  measure  goals 

technical  performance  measures 

schedule  risks 

technical  risks 

tnne-to-complete 

workhours  (by  labor  category) 

These  estimating  categories  are  typical,  however,  each  project  has  its  own  peculiar 
requirements  which  may  add  to  or  subtract  from  this  list.  Notice  that  costs  are  usu¬ 
ally  estimated  first  as  targets  and  that  actual  costs  are  tracked  separately,  this  is  a 
useful  mechanism  for  all  resources  that  must  be  tracked  and  reported  to  an  outside 
agency.  Also  notice  that  “time-to-complete”  is  a  resource  that  is  virtually  impossible 
to  accumulate  without  a  technical  fiow/sequenco  diagram  or  network  to  determine 
critical  path  relationships.  Labor  category  workhour  estimates  are  not  useful  in 
themselves,  they  are  used  for  “resource  leveling"  by  the  technical  manager.  (Resource 
leveling  trades  off  available  manpower/talent  against  schedule,  therefore,  knowledge 
of  the  critical  path  is  necessary.)  The  risk  estimates  may  be  combined  to  create  new 
estimating/tracking  categories  such  as  “most  likely  cost,”  but  they  are  most  useful 
when  used  in  establishing  estimated  targets. 

The  many  relationships  which  are  possible  between  estimating  categories  and 
for  accumulating  estimated  resources  are  difficult  to  maintain  manually,  even  on 
small  projects.  Typically,  a  spreadsheet  is  used  to  display  and  maintain  the  estimat 
ing  categories  and  the  estimated  quantities  Popular  electronic  spreadsheets  allow 
building  these  complex  relat.onslups  into  the  spreadsheet  cell  relationships,  this 
effectively  automates  the  maintenance  of  the  estimates  once  the  original  estimates 
are  made  Project  expenditure  data  can  be  dumped  into  the  spreadsheet  and  auto¬ 
matically  compared  to  the  estimates  to  highlight  potential  problem  areas.  The  presen 
tation  of  the  estimates  for  any  one  estimating  category  keyed  tv  the  WBS  is  often 
required  by  contracts  or  work  agreements,  most  typical  examples  are  cost  work 
breakdown  structures  of  various  sorts 
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Abbreviated  examples  of  the  two  basic  types  of  work  breakdown  structure 
are  shown  in  figures  XXII-2  and  XXII-3. 


LEVEL  1  XYZ  COMMUNICATION  SYSTEM 

cgOI 

LEVEL  2  TRANSMITTER  RECEIVER  ANTENNA 

GRP  (PROJECT  TASKS) 
cgOI.10 

LEVEL  3  EXCITER  PWRAMP 
cgOl.10.1  cg01.10.2 

LEVEL  4  TUNER  MODULATOR 
cgOt. 10.1.1  cgOI  10.1.2 

LEVEL  5  FREQ  SYNTH  REMOTE  CONTROL 
cgOi.10  ill  cgOI. 10.1. 12 

LEVEL  6  VCO  PHASE  CONTROL  CONTROL  FILTER 
cgOI. 10.1. 1.1  1  cgOI  101  11.2  etc. 

LEVEL  7  E  DESIGN  M  DESIGN 

cgOI. 10  lllle  cgOI  101.1  1  l.m 


Figure  XXII-2  A  partial  sample  task-oriented  WBS 

t 


LEVEL  1 

XYZ  COMMUNICATION  SYSTEM 

cgOI 

LEVEL  2  M  DESIGN 
T&E  .... 

E  DESIGN  PROGMGT 

SYS  ENGR 

cgOl.m  etc. 

LEVEL  3  DESIGNDRAFTING  FABRICATION 

ASSEMBLY 

cgOl.m. D  cgOl.m  d 

etc. 

Figure  XXIU3.  A  partial  functionally  oriented  WBS. 
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Table  XXH-2.  Project  tasks  from  MILSTD-881 
(expanded  version  showing  PM/SE  elements). 

1  Program/System  Management 

1.1  Program  Management 

1.1.1  Management  Staff 

1.1.2  Management  Reports 

1.1.2.1  Financial  Reports 

1.1.2.2  Progress  Reports 

1.1.3  Management  Information  System  (use  and  maintenance) 

1. 1.3.1  Work  Breakdown  Structure 

1. 1.3.2  Schedule/Milestone  Chart 

*  1. 1.3.2. 1  Milestone  0 

*  1.1. 3.2.2  Milestone  1 

*  1.1.3.2.3  Milestone  2 

*  1.1. 3.2.4  Milestone  3 

1. 1.3.3  Technical  Flow/Sequence  Network 

1. 1.3.4  (etc.) 

1.1.4  Travel 

1.1.5  Presentations 

1.1.6  Team  (staff)  Training 

1.1.7  Contractor/Vendor  Liaison 

1.1.8  Project  Master  Plan  (including  Proposal  Preparation) 

1.1.9  Vendor  Proposal  Evaluations 

1.2  System  Engineering 

1.2.1  System  Engineering  Management 

1.2. 1. 1  System  Engineering  Staff 

1.2.1.2  Risk  Management 

1.2. 1.3  Technical  Management  Information  System 

1.2.1.3.1  Technical  Performance  Measure 
Tracking 

1.2.1.3.2  Production  Cost  Tracking 

1.2.1.3.3  Life-Cycle  Cost  Tracking 

1.2.1.3.4  Specification  Tree 

1.2.1.3.5  Documentation  Tree 

1.2.1.4  Contractor/Vendor  Reviews 

1.2. 1.5  System  Engineering  Management  Plan 

1.2.2  System  Technical  Direction 

1.2.2.1  Requirements  Analysis 

X  1.2. 2. 1.1  Affordability  Analysis 

1.2.2.1.2  Mission  Analysis 

1.2.2.1.3  System  Trade  Studies 

1.2.2.1.4  System  Design 

1.2.2. 1.4.1  Conceptual  Design 

1.2.2.1.4.2  Risk  Assessment 

1.2.2.1.4.3  Cost  Assessment 

1.2.2.1.4.4  Feasibility  Demonstra¬ 
tion 
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SRR 

SDR 

OSR 

PDR 

CDR 

TRR 

FCA 

PCA 

FQR 


Table  XXXI-2.  Project  tasks  from  MIL-STD-881 
(expanded  version  showing  PM/SE  elements)  (continued). 

1.2.2.1.4.4.1  Explora¬ 
tory  Devel¬ 
opment 

1.2.2.1.4.5  Minimum  Performance 
Specifications 

1.2.2.1.4.6  Technical  Performance 
Measures 

1.2.2.2  System  Requirements 

1.2.2.2.1  System  Specification  (MIL-STD-490 
TYPE  A) 

1.2.2.2.2  Requirements  Flowdown/Traceability 

1.2.2.2.3  Performance  Verification 

1. 2.2.2. 3.1  Advanced  Development 

1.2.2.2.4  Design  Reviews 

1.2.2.2.4.1  System  Requirements 
Review 

1.2. 2. 2.4.2  System  Design  Review 

1.2.2.2.4.3  Development  Specifica¬ 
tion  Reviews 

1.2.2.2.4.4  Preliminary  Design 
Review 

1.2. 2.2.4. 5  In-process  Reviews 

1.2. 2. 2.4.6  Critical  Design  Review 

1.2.2.2.4.7  Test  Requirements 
Review 

1.2.2.2.4.8  Functional  Configuration 
Audit 

1.2.2.2.4.9  Physical  Configuration 
Audit 

1.2.2.2.4.10  Formal  Qualification 
Review 

1.2. 2. 2. 5  Change  Control  Board 

1.2. 2. 2. 6  Quality  Program 

1.2.2.3  System  Integration 

1.2.2.3.1  Interface  Specifications  &  Design 
Control 

1.2.2.3.2  Intorsystem/Intrnsystem  Compatibility 
Assurance 

1.2.2.3.3  Electromagnetic  Compatibility  (EMC) 

1.2.2.3.4  TEMPEST 

1.2.2.3.5  Technical  Usage/Environmental  Crite¬ 
ria 

1.2.2.3.6  Platform  Integiation/Installation 
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Table  XXI1-2.  Project  tasks  from  MIL-STD-881 
(expanded  version  showing  PM/SE  elements)  (continued). 

1.2.2.4  Software  System  Design 

1.2.2.4.1  Software  Requirements  Allocation 

1.2.2.4.2  Hardware/Software  Interface  Control 

1.2.2.4.3  Software  Performance  and  Interface 
Specifications 

1.2.2.4.4  Software  Documentation  Audit 

1.2.2.4.5  Software  Design  Reviews 

1.2.2.5  Mission  Operations  Planning 

1.2.2.5.1  Operations  Plan  Generation 

1.2.2.5.2  Employment  Procedures 

COE  1.2.2.5.2.1  Concept  of  Employment 

1.2.2.5.3  Facility  Support  Requirements 

1.2.2.6  System  Effectiveness 

1.2.2.6.1  Reliability 

1.2.2.6.2  Availability 

1.2.2.6.3  Maintainability 

1.2.2.6.4  Safety 

1.2.2.6.5  Logistics/LSA/ILS/LOR 

1.2.2.6.6  Contamination  Control 

1.2.2.6.7  Transportability  (PHST) 

1.2.2.6.8  Human  Factors  Engineering 

1.2.2.6.9  Personnel  and  Training 

1.2.2.6.10  Parts,  Materials,  and  Processes 

1.2.2.6.11  Testability 

1.2.2.7  System  Test  Planning  and  Audit 

TEMP  1.2.2.7.1  System  Test  and  Evaluation  Master 

Plan 

1.2.2.7.2  Design  for  Test  Criteria 

1.2.2.7.3  Design  Analysis  Review 

1.2.2.7.4  Test  Procedure  Review 

1.2.2.7.5  Design  Audit 

1.2.2.7.6  Test  Data  Review 

1.2.2.7.7  Test  Support 

2  Documentation 

2.1  Technical  Publications 

2.2  Engineering  Data 

2.3  Management  Data 

2.4  Support  Data 

2.5  Data  Depository 

3  System  Test  and  Evaluation 

3.1  Developmental  Test  and  Evaluation 

3.1.1  DTI 

3.1.2  DT IIA 

3.1.3  DT  I  IB  (TECHEVAL) 

3.1.4  DT  III  (FIRST  ARTICLE) 
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Table  XXII-2.  Project  tasks  from  MIL-STD-881 
(expanded  version  showing  PM/SE  elements)  (continued). 

3.2  Operational  Test  and  Evaluation 

3.2.1  OT I 

3.2.2  OT  IIA 

3.2.3  OT  IIB  (OPEVAL) 

3.2.4  OTIII 

3.3  Mockups  and  Simulations 

3.3.1  Mockups 

3.3.1. 1  Human  Factors  Mockup 

3.3.2  Simulations 

3.4  T&E  Facilities 

3.5  T&E  Support 
Training 

4.1  Services 

4.1.1  Interim 

4.1.2  Operational 

4.2  Facilities 

4.2.1  Interim 

4.2.2  Operational 

4.3  Equipment 

4.3.1  Interim 

4.3.2  Operational 
Peculiar  Support  Equipment 

5.1  Operational/Intermediato  Level 

5.2  Depot  Lovel 
Common  Support  Equipment 

G.l  Operational/Intermcdiate  Level 

G.2  Depot  Level 

Facilities 

7.1  Maintenance 

7.2  Equipment  Acquisition/Modernization 

7.3  Construction/Conversion/Expansion 
Initial  Spares/Repair  Parts 

8.1  (for  Subsystem  10) 

8.2  (for  Subsystem  20) 


System  Implementation 

9.1  Platform  Conversion 

9.2  Installation  Kits 

9.3  Installation  and  On-site  Checkout 
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Table  XXH- S.  Elements  ticked  by  a  work  breakdown  structure. 
(May  be  tracked  m  Spreadsheet  Form). 


PROJECT  MANAGEMENT/CONTROI.  ELEMENTS  (13  or  more) 


Cost-to-completo 

Target 

Bid 

Actual 


Worltliours 

Target 

Bid 

Actual 

Schedule  (Calendar  time) 
Target 
Bid 
Actual 


(Development/Acquisition  Cost) 

(Note:  Acquisition  Costs  may  be  broken 
down  into  Development,  Procurement,  and 
Installation/Implementation  Costs- each  ele¬ 
ment  separately  tracked  by  target  and 
actual) 


(Note:  “Target"  represents  tho  expected 

includes  contingency  estimates, 
Actual  is  the  experience  for  the  progress 
achieved.) 


Risk  Factor 

Recovery  (“Fix-it")  Risk  Factor 
Recovciy  Cost  Factor 
Recovery  Schedule  Factor 

SYSTEM  ACQUISITION  ELEMENTS  (9) 

Production  Quantity 


Design-to-Cost  (DTC) 
Target 

Minimum  Expected 
Maximum  Acceptable 
Fixed-Cost  Target 
Fixed-Cost  Actual 
Base  Cost  Target 
Base  Cost  Actual 


(unit  production  cost) 


(Note:  “Target"  represents  the 
expected  value  at  the  time  the 
plan  was  promulgated,  “Actual" 
represents  tho  currently  expected 
value  here  and  below.) 


SYSTEM  ENGINEERING  INFORMATION  ELEMENTS  (14  or  more) 

Technical  Performance  Measure  (TPM) 

Specification  Requirement 
Minimum  Acceptable 
Actual 

Variance  Limit 
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Table  XSH-8.  Elements  tracked  by  a  work  breakdown  structure. 
(May  be  tracked  in  Spreadsheet  Form)  (continued). 

Life-Cycle  Costs  (LCC)  =  Acquisition  Costs  +  LCSC 

Life-Cycle  Support  Costs  (LCSC) 

Nonrecurring  Cost  Target 
Nonrecurring  Cost  Actual 
Recurring  Cost  1  arget 
Recurring  Cost  Actual 

Recurrence  Frequency  Factor  Target  (such  as  Failure  Rate) 
Recurrence  Frequency  Factor  Actual 
LCSC  Target 
LCSC  Actual 


CONTROLLING  THE  PROJECT 


Project  management  involves  a  continuing  effort  to  plan,  organize,  monitor, 
and  control.  The  planning  is  accomplished  through  the  generation  and  maintenance 
of  the  PMP,  which  serves  to  document  the  plan.  Organization  occurs  as  the  plan  is 
implemented  by  assigning  resources  and  responsibilities  to  specific  task  leaders.  The 
PMP  also  documents  the  organization  decisions.  Monitoring  is  accomplished  through 
the  management  information  systems,  the  reports  and  data  produced  by  project 
efforts,  and  by  the  personal  communications  with  task  leaders  and  other  project  par¬ 
ticipants.  The  PMP  spells  out  the  monitoring  and  reporting  procedures  used  in  the 
project.  Control  is  the  “quality  assuranco’  aspect  of  management.  The  delegation  of 
responsibility  and  authority  must  also  mean  accountability.  The  control  effort  main¬ 
tains  this  accountability.  There  are  too  many  details  for  the  oroject  manager  tt  „are 
for  them  all;  therefore,  good  project  control  mechanisms  implemented  in  the  project's 
management  information  systems  allow  only  significant  deviations  from  the  plan  to 
be  flagged  for  the  project  manager’s  attention. 

Technical  performance,  cost,  and  schedule  are  related  to  each  other.  It  is  not 
possible  to  tightly  control  all  three  simultaneously  because  the  control  of  any  two  will 
determine  the  limits  for  the  third.  Figure  XXII-4  illustrates  the  typical  functional  re¬ 
lationship.  Notice  that  there  is  an  optimum  planning  point  for  any  given  project  (the 
minimum  cost/schedule  point  for  the  specified  performance).  In  practice,  it  is  nearly 
impossible  to  determine  this  optimum  point;  however,  it  is  possible  to  plan  into  the 
zone  around  this  point  through  good  estimation  practices.  Simply  because  this  opti¬ 
mum  zone  is  known  does  not  mean  that  the  sponsorship  bureaucracy  will  allow  the 
project  manager  to  actually  propose  and  execute  the  program  in  this  zone.  There  are 
a  host  of  political  operating  factors  which  may  dictato  schedules  or  distort  funding 
profiles,  resulting  in  a  less  efficient  plan. 
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PROJECT  CHANGE  CONTROL 


Any  time  a  plan  is  established,  a  decision  is  made,  or  a  design  is  documented, 
it  costs  in  order  to  effect  a  change.  Moreover,  the  later  a  change  occurs,  the  more  it 
costs.  This  is  true  for  design  changes  and  project  plan  changes  alike.  As  a  result,  it  is 
important  to  manage  chanm-3  The  need  for  change  must  be  recognized  early,  and 
appropriate  decisions  m  >s'  ,  j  made  correctly  and  documented  thoroughly.  The  man¬ 
ager  must  decide  when  a  uiange  is  necessary  and  avoid  unnecessary  changes;  like¬ 
wise,  the  manager  needs  to  recognize  when  a  change  decision  should  be  made.  If  a 
change  is  implemented  too  late,  unnecessary  change  costs  will  be  incurred;  if  a 
change  is  made  before  circumstances  are  stable,  additional  change  upon  change  will 
result.  Change  is  inevitable,  so  plans  must  include  contingencies  for  change.  Risk 
assessraent/risk  management  techniques  greatly  assist  the  project  manager  or  sys¬ 
tems  engineer  to  manage  change.  Configuration  management  techniques  can  be 
applied  to  the  project  plans  as  well  as  the  system  designs  to  help  control  changes. 
Effective  management  information  reporting  is  necessary  to  detect  the  potential  need 
for  change.  As  a  rule-of-thumb,  the  costs  associated  with  a  chance  will  grow  bv  an 
order-of-magnitude  each  time  the  project  transitions  from  one  phase  to  the  next: 

100%-2QQ%  is  added  to  change  costs  each  time  an  associated  milestone  is  reached 

without  resolving  the  issue. 

ORGANIZATIONAL  CONTROL 

Organizational  control  is  effected  through  leadership  more  than  management 
controls.  While  it  is  important  to  have  the  agreements  and  contracts  in  place  to  set 
the  ground  rules,  the  work  must  be  accomplished  through  people;  people  are  more 
effective  and  productive  when  thoy  are  led  os  part  of  a  team  instead  of  being  managed 
like  an  inort  asset.  The  projoct  manager  can  effect  control  of  the  project  team  well 
beyond  the  projoct  charter  authority  by  following  good  leadership  principles  and  by 
being  a  model  for  the  team  members. 

Projoct  leadership  is  implemented  to  personal  contact.  It  is  difficult  to  lead 
from  an  isolated  ivory  tower.  Regular  project  meetings  can  be  an  important  forum  for 
exercising  organizational  control.  Likewise,  regular  contact  with  the  project  “worker 
bees”  by  making  periodic  visits  to  their  work  sites  contributes  to  the  control  exer¬ 
cised  by  the  project  leadership.  It  is  the  responsibility  (and  in  the  best  interests)  of 
the  projoct  managor  and  systems  engineer  to  create  as  favorable  a  work  environment 
for  the  task  leaders  and  the  project  workers  under  their  control.  While  it  may  not  be 
possible  to  remodel  an  office  or  build  a  now  building  simply  to  provide  this  environ¬ 
ment,  it  is  usually  possible  to  improve  the  work  environment  by  contributing  new 
tools,  tost  equipment,  and  other  things  to  help  the  accomplishing  organizations.  An 
active  system  of  “attaboys"  to  the  good  performers’  supervisors  also  helps  create  a 
positive  work  environment.  Generally,  people  like  to  know  where  they  stand  and  how 
their  efforts  contribute  to  the  overall  success  of  the  project.  These  steps  of  leadership 
greatly  affect  the  required  organizational  control. 

On  those  occasions  when  a  person  or  organization  fails  to  perform  satisfacto¬ 
rily,  the  project  manager  should  take  immediate  steps  to  remedy  the  situation. 
Although  a  good  project  manager  can  lead  ordinary  people  to  do  extraordinary  things, 
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some  project  participants  may  insist  on  being  “bad  apples.”  The  project  manager  nor¬ 
mally  should  try  to  correct  the  situation  through  personal  disciplinary  measures  first, 
but  should  insist  that  the  person  or  organization  be  replaced  if  the  unacceptable  per¬ 
formance  continues. 

When  contractual  support  is  used,  the  project  manager  must  implement  organ¬ 
izational  controls  through  the  contract.  This  means  that  the  contract  must  be 
planned  to  have  those  controls  in  place  at  the  time  it  is  issued  and  that  the  methods 
of  organizational  performance  measurement  are  clearly  spelled  out.  The  PMP  and  the 
associated  procurement  plans  provide  adequate  justification  for  the  selection  of 
appropriate  contract  clauses  to  maintain  control.  The  manager  wants  to  avoid  an 
adversarial  relationship  with  the  contractors,  but  this  is  usually  more  feasible  when 
the  contract  has  good  organizational  control  “teeth.” 

A  similar  situation  applies  to  work  agreements  and  taskings  to  other  organiza¬ 
tions.  Also,  the  project  manager  must  have  contigency  plans  for  the  failure  of  an 
organization  to  perform.  These  contingency  plans  may  or  may  not  be  part  of  the  for¬ 
mal  PMP.  Sometimes  a  Government  organization  is  “the  only  game  in  town"  as  far  as 
accomplishing  a  specific  task.  The  effective  project  manager  can  still  elicit  coopera¬ 
tion  and  gain  effective  organizational  control  by  understanding  the  bureaucracy  asso¬ 
ciated  with  the  organization  and  tailoring  the  project  requirements  to  fit  it  as  closely 
as  possible.  Where  this  is  not  possible,  the  active  role  of  the  organization  in  project 
planning  can  be  used  to  elicit  cooperation  because  the  justifications  for  the  taskings 
will  become  apparent  to  the  performing  organization. 

COST/SCHEDULE  CONTROLS 

Many  elaborate  systems  have  been  developed  for  tho  control  of  project  costs 
and  schedule.  Some  of  tho  systems,  used  widely  on  large  military  contracts,  are  as  fol¬ 
lows: 


Contract  Performance  Report  (CPR)  System 

Contract  Funds  Status  Reporting  (CFSR)  System 

Cost/Schcdulo  Status  Reporting  System  (C/SSRS) 

Uniform  Cost  Accounting  and  Reporting  System  (UCARS)  (see  MIL- 
STD-1260) 

All  of  these  systems  are  reporting  data  intensive  and  assume  automated  accounting 
systems  which  are  set  up  to  report  in  DOD  formats  according  to  standard  military 
contract  requirements.  They  are  not  universally  appropriate.  However,  they  each  il¬ 
lustrate  the  common  requirement  for  cost  and  schedule  control  —  costs  must  he 
reported  as  a  function  oLwork-Scheduled  and  work  performed.  Figure  XXII-5  shows  a 
sample  cost  summary  trend  chart  from  DI-A-2011.  By  comparing  the  actual  costs,  the 
budgeted  cost  for  work  scheduled,  and  the  budgeted  cost  for  work  performed,  the  pro¬ 
ject  manager  can  see  if  the  right  amount  of  work  is  being  accomplished  for  the 
amount  of  funds  expended  at  any  given  time.  These  reports  are  generated  for  each 
work  package,  so  there  can  be  a  lot  of  reporting  data.  The  project  manager  is  really 
only  interested  in  those  tasks  which  are  behind  in  accomplishing  the  work  and  ahead 
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Figure  XXII-5.  Sample  cosl/scPedule/control. 


of  budgeted  expenditures.  These  are  the  tasks  that  demand  management  attention  to 
correct  whatever  is  not  going  according  to  plan.  By  working  with  the  task  managers 
during  the  planning  phase,  realistic  reporting  methods  can  be  designed  to  reduce  the 
reporting  burden  on  the  task  manager  while  effectively  highlighting  the  tasks  which 
need  management  attention. 

Another  method  of  effective  cost  and  schedule  control  is  to  plan  the  project 
with  short-term  milestones  (2  weeks  to  2  months  apart).  While  this  creates  a  plan¬ 
ning  burden,  the  cost  and  schedule  control  is  effectively  reduced  to  meeting  mile¬ 
stones.  This  method  works  reasonably  well  with  highly  modular  tasks,  such  as  some 
software  design  tasks. 

The  schedule  should  be  watched  closely  over  a  horizon  of  no  more  than  6 
months,  and  preferrably  over  a  shorter  period.  Each  active  task  should  have  at  least 
one  reportable  milestone  in  the  horizon  period.  Since  the  project  is  planned  with 
major  milestones  and  then  intermediate  milestones  (with  projected  costs),  these 
become  a  baseline  plan.  Each  task  manager  should  periodically  (monthly,  bimonthly, 
quarterly)  provide  a  detailed  task  plan  which  covers  the  horizon  period  plus  the  inter¬ 
val  between  plan  updates.  This  keeps  the  planning  and  reporting  efforts  to  a  mini¬ 
mum  and  still  provides  timely  cost  and  schedule  control. 

Whatever  method  of  cost  and  schedule  reporting  is  selected,  the  project  man¬ 
ager  uses  an  allowed  variance  from  the  budgeted  cost  for  work  scheduled  to  trigger 
management  attention.  A  practical  variance  for  a  significant  task  is  2  to  5  percent. 
Larger,  riskier,  and  more  important  tasks  merit  tighter  variance  limits  that  small, 
relatively  unimportant  tasks. 

TECHNICAL  PERFORMANCE  CONTROL 

Chapter  IV  discusses  performance  measures  as  a  step  in  system  design  and  as 
a  method  of  contract  evaluation.  The  individual  specification  parameters  that  become 
part  of  the  system  specification  are  usually  a  composite  of  many  technical  parameters 
that  have  been  combined  through  systems  analysis  and  design  tradeoffs.  As  the  sys¬ 
tem  is  partitioned,  these  system  design  parameters  must  be  allocated  to  the  system 
elements  which  result.  The  parameter  values  allocated  to  partitioned  elements 
become  Technical  Performance  Measures  (TPMs).  These  TPMs  exist  at  the  smallest 
system  partition;  therefore,  they  can  be  correlated  to  the  product  WBS,  which  also 
reflects  this  system  design.  More  than  one  TPM  may  exist  at  the  work  package  level 
of  the  WBS,  but  the  system  design  shows  how  the  TPMs  ore  related.  As  design  pro¬ 
gresses,  as  parts  are  selected,  and  as  tests  are  performed,  the  measured  or  projected 
value  of  the  TPMs  varies.  But  as  the  designer  works  toward  task  completion,  the 
TPMs  the  designer  is  responsible  for  should  converge  toward  the  specification 
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WBS  TITLE  WBS  NO.  PARAMETER  NAME 


O  2  O 


M 


the  TPM  is  correlated  to  the  WBS  and  to  the  schedule  (calendar) 

the  requirement,  projected  value,  actual  value  and  variance  limits  are  all  read¬ 
ily  visible 

In  practice,  most  TPMs  have  only  a  single  variance  limit  which  converges  on  the 
specified  viue  at  task  completion.  The  project  manager  and  systems  engineer  are 
only  interested  in  tasks  which  report  actual  values  outside  of  the  variance  limits. 
(Anything  else  will  meet  performance  requirements).  When  a  TPM  goes  out  of  limit, 
the  systems  engineer  takes  action  to  bring  the  TPM  within  limits.  This  may  involve  a 
focus  of  management  attention  or  engineering  talent  on  the  problem,  or  it  may  mean 
tweaking  the  system  design  to  reallocate  performance  within  the  system  partitions. 
As  with  the  cost  and  schedule  controls,  the  planning  of  the  allowed  variance  limitfe) 
depends  on  the  significance  of  the  TPM  to  the  overall  system  design.  In  no  case 
should  the  variance  allow  less  than  minimum  acceptable  performance. 

RISK  CONTROL 


Risk  control  amounts  to  “stamping  out  forest  fires"  while  they  are  still  smol¬ 
dering  rather  than  after  they  get  out  of  control.  Risk  control  is  an  important  aspect  of 
risk  management  (see  chapter  21).  Cost,  schedule,  and  technical  risks  are  effectively 
controlled  through  adequate  planning  (including  contingency  planning),  good  commu¬ 
nications  of  project  status  (including  an  effective  management  information  system), 
and  timely  action  in  addressing  problems.  The  timeliness  of  action  is  often  dependent 
on  the  setting  of  variance  limits  for  the  management  system  reports  that  can  trigger 
management  attention.  High  risk  priority  tasks  (see  Table  XXI- 1)  should  have 
tighter  variance  limits  than  lower  risk  priority  tasks.  A  cost  variance  limit  of  2  per¬ 
cent  of  the  total  task  expenditure  for  the  planning  horizon  (usually  6  months  —  see 
figure  22-7)  is  usually  appropriate  for  tasks  with  a  risk  priority  of  1  through  4.  A  cost 
variance  limit  of  5  percent  is  usually  appropriate  for  tasks  with  a  risk  priority  of  5 
through  8,  a  cost  variance  limit  of  10  percent  is  usually  workable  for  tasks  with  a  risk 
priority  o'  9  or  10.  These  limits  may  be  a  crusted  for  unusual  tracking  requirements, 
but  they  must  also  be  adjusted  for  the  availability  of  contingency  funds.  The  lack  of 
contingency  overhead  in  any  project  plan  artificially  increases  the  risk. 

For  technical  risks,  the  variance  limits  should  be  tightened  at  a  rate  that  will 
expose  a  parameter  that  is  not  converging  on  its  specification  requirement  in  time  to 
effect  a  design  change,  if  that  is  required.  The  timing  of  this  threshold  will  depend  on 
the  dependencies  and  interdependencies  between  tasks  as  well  as  the  specification 
contingencies  built  into  the  system  design  (intentional  overspecification  to  provide 
room  for  underdesign). 

In  all  cases,  risk  control  is  greatly  enhanced  by  timely,  open,  and  honest  com¬ 
munications  between  the  task  leaders  and  the  systems  engineer,  project  financial 
manager,  and  the  project  panager. 

Project  political  risks  require  an  entirely  different  set  of  controls.  As  with 
other  project  risks,  political  risks  require  an  effective  management  information  sys¬ 
tem,  except  that  this  is  an  intelligence  system  for  political  risks.  Political  risk  strate¬ 
gies  mostly  involve  avoidance  or  evasion.  Both  strategies  require  expert  knowledge  of 
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the  persons  in  the  administrative  and  project  chains  of  command  (anybody  who  might 
have  review  or  approval  authority  over  any  aspect  of  the  project)  and  of  the  rules  mid 
uflderwhl<;h  each  person  operates.  THESE  ARE  MINIMUM  REQUIRE¬ 
MENTS.  In  addition,  it  is  helpful  to  know  the  personality  quirks  of  the  key  people 
and  their  personal  perferrences  as  they  relate  to  the  project.  If  the  project  manager  or 
the  project  sponsorship  can  develop  good  working  relationships  and  personal  friend¬ 
ships  with  these  key  people,  it  is  all  the  better.  Political  risk  control  also  requires  a 
detailed  knowledge  of  the  competition  (projects  competing  for  the  same  budgetary 
resources  or  addressing  similar  operational  requirements  or  developing  a  competing 
technology).  This  knowledge  must  sometimes  be  developed  through  cooperation  and 
sharmg  of  information  with  the  competing  project.  (It  is  important  not  to  undermine 
the  competition;  afterali,  we  are  all  on  the  same  side  developing  systems  to  serve  the 
defense  of  our  nation.  The  overall  success  does  NOT  require  the  success  of  our  pro¬ 
ject,  but  the  success  of  each  project  should  enhance  the  broadest  goal  of  the  security 
of  our  nation.)  The  intelligence  gained  by  the  project  manager  allows  timely  decisions 
and  actions  to  be  taken  to  avoid  negative  issues  and  their  impact.  Sometimes  compet¬ 
ing  projects  can  cooperate  in  their  political  intelligence  and  develop  a  mutual 
approach  which  enhances  both  projects.  In  almost  all  cases,  the  personal  integrity  of 
the  project  manager  can  influence  the  reputation  of  the  project. 

APPLICATION  OF  SUPPORTING  “ILITY”  TASKS 

Most  of  the  military  standards  covering  the  support  engineering  tasks  have 
'm  k°D  ™?trlce?  wWch  show  when  (in  which  program  phase)  the  various  tasks 
should  be  performed.  MIL-STD-2107  covers  many  of  the  topics  which  are  not  easily 
determined  from  the  supporting  standards  and  specifications.  Table  XXII-4  provides  a 
breakout  of  support  engineering  tasks  and  provides  an  application  matrix  for  these 
tasks.  A  survey  of  Table  XXII-4  shows  that  many  of  the  tasks  must  be  tailored;  the 
project  planner  is  exhorted  to  include  expertise  for  these  support  engineering  areas  in 
the  generation  of  project  plans.  Also  see  figure  XXII-7.  s 
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Table  XXH-4.  Support  task  application  matrix. 
CONFIGURATION  MANAGEMENT  REQUIREMENTS  PER  MIL-STD-480B 


C  D  F  P  0 

0  &  S  R  & 

N  V  D  O  S 

100  PROGRAM  CONTROL 
A  A  A  A  A  Ifll  PM  PT  AN 

101.1  CHANGE  PROCEDURES  AND  AUTHORITIES 

101.2  INTERFACE  CONTROL 

101.3  REVIEWS  AND  AUDITS 

T  A  A  A  T  102  CONFIGURATION  MANAGEMENT  RESOURCES 

102.1  CM  TEAM  MEMBERS 
102  2  SUBCONTRACTOR  CM 

102.3  AUTOMATED  TOOLS 
C  C  A  A  C  103  CONFIGURATION  CONTROL  BOARD 

200  BASELINE  IDENTIFICATION 
A  A  A  A  I  201  FUNCTIONAL  BASELINE 
AAAI  202  ALLOCATED  BASELINE 
A  A  A  203  PRODUCT  BASELINE 


C  C  A  A  A 

C  C  A  A  A 

C  A  A  A 

C  A  A  A 

A  A  A  A  A 


300  STATUS  ACCOUNTING 

301  DATABANK  ESTABLISHMENT 

302  LIBRARY  INDEXING 

303  STATUS  REPORTING 

304  FOLLOW-ON  AUDITS 

305  PROPOSED  CHANGE  STATUS  TRACKING 

305. 1  DESIGN  IMPACT  EVALUATIONS 

305.2  QUALITY  EVALUATIONS 
305  3  SAFETY  EVALUATIONS 

305.4  COST  EVALUATIONS 


LOGISTICS  SUPPORT  ANALYSIS  TASKS  PER  MIL-STD-1388-1A 


C  D  F  P  O 
O  &  S  R  & 

N  V  D  O  S 

100  PROGRAM  PLANNING  AND  CONTROL 

A  A  T  101  EARLY  LSA  STRATEGY  (Describes  proposed  supportabihty  objectives 

and  tasks  required  to  achieve  those  objectives.  Becomes  part  of  the 
overall  project  acquisition  strategy.) 

101.1  CONCEPT  UAL  LSA  (Part  of  Functional  Baseline  at  SRR) 

101.1.1  IDENTIFY  MISSION  AND  FUNCTIONAL 
REQUIREMENTS 

10 1. 1.2  IDENTIFY  PROGRAM  CONSTRAINTS  (fund¬ 
ing,  schedule,  available  personnel,  strategic 
materials,  and  other  key  resources) 

101.1.3  IDENTIFY  DATA  BASES  available  for  use  in 
supporting  LSA  tasks 

101.1.4  IDENTIFY  DELIVERY  MILESTONES  FOR 
LSA  PRODUCTS 

101.1.5  IDENTIFY  ANALOG  SYSTEMS/E  '  jIPMENT 
available  to  support  LSA 

101.1.6  TAILOR  LSA  TASK  REQUIREMENTS  TO  FIT 
PROGRAM  CONSTRAINTS 
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Table  XXII -4.  Support  task  application  matrix  (continued). 


101.2  ALLOCATED  BASELINE  UPDATE  (at  SDR/MS  I) 

101.3  DEVELOPMENT  BASELINE  UPDATE  (at  PDR/MS II) 

101.4  DESIGN  CONFIGURATION  UPDATE  (at  CDR) 

101.5  PRODUCT  CONFIGURATION  UPDATE  (at  MS  III) 

A  A  A  A  102  LSA  PLAN  (describes  tasks  and  milestones  and  interfaces  to  other 

project  activities)  (May  be  part  of  another  plan,  such  as  an  ILSP, 
ALP.  PMP,  etc.) 

102.1  GENERATE  LSA  PLAN 

102.1.1  IDENTIFY  LSA  TASKS  (Tailor  from  MIL- 
STD-I388-1A) 

102. 1.2  IDENTIFY  PROJECT  TEAM  PARTICIPANTS 
AND  ORGANIZATIONAL  RELATIONSHIPS 

102. 1.3  SPECIFY  DESIGN  TEAM  PARTICIPATION 

102.1.4  RELATE  TO  OTHER  PROJECT  PLANS 

102.1.5  IDENTIFY  MILESTONES  AND 
DELIVERABLES 

102  2  ALLOCATED  BASELINE  UPDATE  (at  SDR/MS  I) 

102  3  DEVELOPMENT  BASELINE  UPDATE  (at  PDR/MS  II) 

102.4  DESIGN  CONFIGURATION  UPDATE  (at  CDR) 

102.5  PRODUCT  CONFIGURATION  UPDATE  (at  MS  III) 

A  A  A  A  103  PROGRAM  AND  DESIGN  REVIEWS  (Defines  which  LSA  elements 

are  to  be  reviewed  at  project  design  reviews,  at  program  reviews,  or 
at  special  LSA  reviews.  Specifies  review  participants,  review  ele¬ 
ments  (content),  special  procedures,  and  schedules.) 

200  MISSION  AND  SUPPORT  SYSTEMS  DEFINITION 

AAA  201  USE  STUDY 

20 1. 1  SUPPORT  ABILITY  FACTORS 

201.2  SYSTEM  EFFECTIVENESS  FACTORS 

201.3  USAGE  ENVIRONMENT  FACTORS 

201.4  USE  STUDY  REPORT  (include  in  Functional  Baseline) 

201.5  ALLOCATED  BASELINE  UPDATE  (at  SDR/MS  I) 

201.6  DEVELOPMENT  BASELINE  UPDATE  (at  PDR/MS  II) 

201.7  DESIGN  CONFIGURATION  UPDATE  (at  CDR) 

201.8  PRODUCT  CONFIGURATION  UPDATE  (at  MS  III) 

A  A  A  T  T  202  STANDARDIZATION  STUDY 

202.1  MISSION  HARDWARE 

202. 1. 1  SUPPORT  ABILITY  CONSTRAINTS 

202.1.2  SUPPORTABILITY  CHARACTERISTICS 

202.1.3  LEVEL  OF  STANDARDIZATION  DECISION 
INPUTS 

202.1.4  DOCUMENTATION  REQUIREMENTS 

202. 1.5  BUILD,  BUY,  MODIFY  DECISION  INPUTS 

202.1.6  RISK  ASSESSMENT 

202.2  MISSION  SOFTWARE 

202.2.1  SUPPORTABILITY  CONSTRAINTS 

202.2.2  SUPPORTABILITY  CHARACTERISTICS 

202.2.3  LEVEL  OF  STANDARDIZATION  DECISION 
INPUTS 

202.2.4  DOCUMENTATION  REQUIREMENTS 

202.2.5  BUILD,  BUY,  MODIFY  DECISION  INPUTS 

202.2.6  RISK  ASSESSMENT 

202.3  SUPPORT  HARDWARE 

202.3.1  SUPPORTABILITY  CONSTRAINTS 

202.3.2  SUPPORTABILITY  CHARACTERISTICS 
202.3-3  LEVEL  OF  STANDARDIZATION  DECISION 

INPUTS 
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Table  XXII-4.  Support  task  application  matrix  (continued). 


AAA 
A  A 
AAA 

A  A 
AAA 

AAA 

A  A 

AAA 

AAA 

A  A  T 


A  A  A  C 


202  3.4  DOCUMENTATION  REQUIREMENTS 

202.3.5  BUILD,  BUY,  MODIFY  DECISION  INPUTS 
202  3.6  RISK  ASSESSMENT 

202.4  SUPPORT  SOFTWARE 

202.4.1  SUPPORTABILITY  CONSTRAINTS 

202.4.2  SUPPORTABILITY  CHARACTERISTICS 

202.4  3  LEVEL  OF  STANDARDIZATION  DECISION 

INPUTS 

202.4.4  DOCUMENTATION  REQUIREMENTS 

202.4.5  BUILD,  BUY,  MODIFY  DECISION  INPUTS 

202.4.6  RISK  ASSESSMENT 

202.5  STANDARDIZATION  CONCEPTS  REPORTS/PLAN 
(incorporated  into  system  design  as  part  of  allocated  base¬ 
line) 

203  COMPARATIVE  ANALYSIS 

203.1  IDENTIFY  COMPARATIVE  SYSTEMS  (see  101.1.5) 

203.2  BUILD  BASELINE  COMPARISON  SYSTEM  (amalgama- 
tion  of  analogs  modified  to  system  design  requirements) 

203  3  ANALYZE  COMPARATIVE  SYSTEM  CHARACTERISTICS 

203.4  IDENTIFY  QUALITATIVE  SUPPORTABILITY  PROB¬ 
LEMS 

203  5  IDENTIFY  SUPPORTABILITY,  COST,  AND  READINESS 
DRIVERS 

203.6  IDENTIFY  UNIQUE  SYSTEM  DRIVERS 

203.7  IDENTIFY  RISKS  AND  ASSUMPTIONS 

203.8  ANALYSIS  REPORT  (include  in  Allocated  Baseline  and 
update  with  system  design) 

204  TECHNOLOGICAL  OPPORTUNITIES  (Using  inputs  from  the  com¬ 
parative  analysis  (203),  identify  and  evaluate  design  opportunities  for 
improvements  in  support  characteristics.  Provide  recommended  sys¬ 
tem  design  specifications.) 

205  SUPPORTABILITY  AND  SUPPORTABILITY  RELATED  FACTORS 
(Using  inputs  from  tasks  202, 203,  and  204,  perform  an  operational 
support  cost  analysis  (part  of  a  Life-cyle  cost  analysis)  and  a  sup¬ 
port  system  effectiveness  analysis  (numerical  analysis  of  suppor- 
tabiiity  impacts  on  operational  availability)  to  determine  quantitative 
supportabibty  factors  and  to  provide  system  design  objectives, 
goals,  constraints,  and  thresholds.  The  results  of  this  analysis  are 
doumented  in  a  report  describing  the  system  design  and  become  part 
of  the  Allocated  Baseline,  being  updated  accordingly.) 


A  A  A  C  C 
A  A  T  C  C 
A  A  T  C  C 


A  A  T  C  C 
T  A  A  C  C 
A  A  A  C  C 


PREPARATION  AND  EVALUATION  OF  ALTERNATIVES 
301  FUNCTIONAL  REQUIREMENTS  IDENTIFICATION 

301.1  DETERMINE  FUNCTIONAL  REQUIREMENTS 

301.2  IDENTIFY  UNIQUE  FUNCTIONAL  REQUIREMENTS 

301.2.1  DETERMINE  CORRECTIVE  MAINTENANCE 
TASKS  (using  results  of  FMECA IAW  M1L- 
STD-1629) 

301.2.2  DETERMINE  PREVENTATIVE  MAINTE¬ 
NANCE  TASKS  (using  results  of  reliability  cen¬ 
tered  maintenance  (RCM)  analysis) 

301.2.3  DETERMINE  OTHER  MAINTENANCE  TASKS 
30  IB  FORMULATE  DESIGN  ALTERNATIVES 

301.4  UPDATE  SYSTEM  DESIGN  AS  REQUIRED 

301.5  UPDATE  ANALYSIS  PRIOR  TO  IDENTIFIED  REVIEWS 


I 


I 
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Table  XXII -4.  Support  task  application  matrix  (continued). 


A  A  A  C  C 
A  A 

A  A  T 

T  T  A  C  C 

T  T  A  C  C 

A  A  A  C  C 

A  A  A  C  C 

A  A  A  C  C 


A  A  A  C  C 

A  A  A  C  C 

AAA 
A  A  T 

A  A  A  C  C 

T  A  A  C 

A  A  T 

A  A  T  C 

A  A  T  C 

A  A  A  C 

A  A 
A  A 
A  A 


302  SUPPORT  SYSTEM  ALTERNATIVES 

302. 1  ANALYZE  TASK  30 1  RESULTS  FOR  SUPPORT  SYSTEM 
ALTERNATIVES 

302.2  MAINTAIN  UP-TO-DATE  CONCEPT  OF  SUPPORT 

302  3  DETERMINE  DESIGN  CONSTRAINTS  FOR  NEW 

DESIGNS  (from  TASK  205) 

302.4  ASSESS  RISKS 

302.5  UPDATE  SUPPORT  PLANS 

303  EVALUATION  OF  ALTERNATIVES  AND  TRADE-OFF  ANALYSIS 

303.1  TRADEOFF  PROCEDURES 

303.1.1  IDENTIFY  SUPPORTABILITY  CRITERIA 

303.1.2  CONSTRUCT/SELECT  APPROPRIATE  ANA¬ 
LYTICAL  MODEL(S) 

303.1.3  EVALUATE  ALTERNATIVES  USING 
MODEL(S)  AGAINST  CRTfERIA 

303.1.4  CONDUCT  SENSITIVITY  ANALYSIS 

303.1.5  ASSESS  PEACETIME  VS  WARTIME 
REQUIREMENTS 

303. 1.6  ASSESS  SUPPORT  SYSTEM  IMPACTS 
303  1.7  ASSESS  RISKS 

303.2  SUPPORT  SYSTEM  TRADEOFFS 

303.3  SYSTEM  DESIGN  TRADEOFFS 

303.4  READINESS  SENSITIVITIES 

303.5  MANPOWER  AND  PERSONNEL  TRADE-OFFS 
303.0  TRAINING  TRADEOFFS 

303.7  LEVEL  OF  REPAIR  ANALYSIS  (see  MIL-STD-1390) 

303.8  DIAGNOSTICS  TRADEOFFS 

303.9  COMPARATIVE  EVALUATIONS 

303.10  ENERGY  TRADEOFFS 

303  11  SURVIVABILITY  TRADEOFFS 

303.12  TRANSPORTABILITY  TRADEOFFS 

305.13  STORAGE  TRADEOFFS 

303.14  PACKAGING.  PACKING,  AND  HANDLING  TRADEOFF'S 


400 

T  A  C  C 
T  A  C  C 

T  A  C  C 
T  A  C  C 
T  A  C  C 
T  A  C  C 
T  A  C 
A  T  C 
T  A  C 
T  A  C 
T  A  C 

A  C 


DETERMINATION  OF  LOGISTIC  SUPPORT  RESOURCE  REQUIRE- 
MENTS 

401  TASK  ANALYSIS 

401.1  ANALYZE  TASKS  RESULTING  FPOM  TRADEOFF 
RESULTS  (TASK  303)  (including  constraints  from  205) 

401.2  DOCUMENT  RESULTS  IN  LSAR  (MIL-STD-1388-2) 

401.3  IDENTIFY  NEW/CRITICAL  REQUIREMENTS 

401.4  RECOMMEND  TRAINING  REQUIREMENTS 

40 1.5  RECOMMEND  DESIGN  IMPROVEMENTS 

401.6  UPDATE  MANAGEMENT  PLANS 

401.7  CONDUCT  TRANSPORTABILITY  ANALYSIS 

401.8  DETERMINE  PROVISIONING  REQUIREMENTS 

401.9  VALIDATE  401.3  REQUIREMENTS  ON  PROTOTYPES 

401.10  UPDATE  LSAR  WITH  DESIGN  CHANGES 

402  EARLY  FIELDING  ANALYSIS 

402.1  ASSESS  INTERIM  ILS 

402.2  ASSESS  ILS  PLANS  AGAINST  EXPERIENCE 

402.3  ASSESS  IMPACT  OF  RESOURCE  SHORT¬ 
FALLS 

402.4  ASSESS  COMBAT  RESOURCE  REQUIRE¬ 
MENTS 

402.5  DEVELOP  PLANS  FOR  PROBLEM  RESOLU¬ 
TION 
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Table  XXII-4.  Support  task  application  matrix  (continued). 


C  403  POST  PRODUCTION  SUPPORT  ANALYSIS  (assess  hfe-cycle  sup¬ 

port  requirements  incorporating  402  results  before  the  production 
line  shuts  down;  prov'  ie  insurance  items  in  suflicient  quantities). 


500 

A  A  A  A  C 
A  A  T 
A  A  T 
A  A  T 
A  T 
A  T 
A  T 
A  T 
ATT 

A  A 


SUPPORT  ABILITY  ASSESSMENT 

501  SUPPORT  ABILITY  TEST,  EVALUATION,  AND  VERIFICATION 

501.1  T&E  STRATEGY 

501.2  T&E  OBJECTIVES  AND  THRESHOLDS 

501.3  UPDATES  AND  CORRECTIVE  ACTIONS 

501.3.1  XDM 

501.3.2  ADM 

501.3.3  EDM 

501.3.4  STM 

501.4  SUPPORTABIL1TY  ASSESSMENT  PLAN  (POST 
DEPLOYMENT) 

501.5  SUPPORT  ABILITY  ASSESSMENT  (POST  DEPLOY¬ 
MENT) 


MAINTAINABILITY  TASKS  PER  MIL-STD470A 


C  D  F  P  O 
O  &  S  R  & 
N  V  D  O  S 

A  A  A  A 
T  A  A  T 
T  A  A  A  T 
T  A  A  T 


PROGRAM  SURVEILLANCE  AND  CONTROL 

101  MAINTAINABILITY  PROGRAM  PLAN 

102  MONITOR/CONTROL  OF  SUBCONTRACTORS/VENDORS 

103  PROGRAM  REVIEWS 

104  DATA  COLLECTION,  ANALYSIS,  AND  CORRECTIVE  ACTiON 
SYSTEM 


T  T  A  C 
T  T  A  C 
TACT 
TACT 

T  A  A  C  T 
TACT 
TACT 


DESIGN  AND  ANALYSIS 

201  MAINTAINABILITY  MODELING 

202  MAINTAINABILITY  ALLOCATIONS 

203  MAINTAINABILITY  PREDICTIONS 

204  FAILURE  MODES  AND  EFFECTS  ANALYSIS  -  MAINTAIN¬ 
ABILITY  INFORMATION 

205  MAINTAINABILITY  ANALYSIS 

206  MAINTAINABILITY  DESIGN  CRITERIA 

207  PREPARATION  OF  INPUTS  TO  THE  DETAILED  MAINTE¬ 
NANCE  PLAN  AND  LOGISTIC  SUPPORT  ANALYSIS  (ISA) 


300 

TACT 


EVALUATION  AND  TEST 

301  MAINTAINABILITY  DEMONSTRATION 


PIIST  PROGRAM  REQUIREMENTS  PER  MIL-STD-1367 


C  D  F  P  O 
O  &  S  R  & 
H-Y-D-P-S 

100 

A  A  A  A  A 


A  A  A  A  C 
A  A  A  A  C 


PROGRAM  CONTROL 

101  INTEGRATION  WITH  OTHER  PROGRAM  ELEMENTS  (system 
engineering,  ILS,  product  assurance,  work  breakdown  structure, 
standardization,  safety,  human  engineering,  etc.) 

102  INCORPORATE  PHST  IN  TECHNICAL/PROGRAM  REVIEWS 

103  ESTABLISH  PROCEDURES  FOR  INCORPORATING  SUBCON¬ 
TRACTORS  IN  THE  PHST  PROGRAM 
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Table  XX1I-4.  Support  task  application  matrix  (continued). 


T  T  A  A  C 

T  A  A  C  C 

T  A  A  C  C 

T  A  C  C 

T  A  C  C 

C  C  A  C  C 

ACC 
T  T  A  C  C 
ACC 

A 


104  PLAN  PHST  TEST  AND  EVALUATION  INTEGRATED  WITH 
PROGRAM  T&E 

PHST  ANALYSIS  TASKS 

201  DETERMINE  PHST  REQUIREMENTS  FROM  LSA  (including 
requirements  for  logistical  and  tactical  movement) 

202  EXTRACT  SPECIAL  STORAGE  AND  STOWAGE  REQUIRE¬ 
MENTS  (need  for  controlled  environments,  special  security  con¬ 
siderations,  cyclic  inspection  requirements,  etc.) 

203  DETERMINE  DIMENSIONAL  AND  WEIGHT  CONSTRAINTS 
FOR  TRANSPORTATION  MODES 

204  ANALYZE  ITEM  DESIGNS  FOR  COMPATIBILITY  WITH  STAN¬ 
DARD  CONTAINERS  AND  PACKAGING  MATERIALS 

205  IDENTIFY  SPECIAL  HANDLING  REQUIREMENTS  (hazardous 
materials,  susceptibility  to  transportation  shock,  special  security 
requirements,  need  for  controlled  environments,  etc.) 

206  IDENTIFY  LIMITED  SHELF  LIFE  ITEMS 

207  IDENTIFY  ITEMS  TO  BE  FORWARD-POSITIONED  RESERVES 

208  DETERMINE  PRODUCT  CLEANLINESS  AND  CONTAMINA¬ 
TION  CONTROL  REQUIREMENTS 

209  PHST  ECONOMIC  ANALYSIS 


300 

A  C 
T  A  C 
A 

ACC 

ACC 

A 

ACC 

T  A  C 

ACC 
T  A  C  C 

A  A  A  C 

A  C 


PHST  DESIGN  TASKS 

301  DETERMINE  ITEM  DESIGN  CHARACTERISTICS  REQUIRED  BY 
PILST  CONSTRAINTS 

302  DETERMINE  SPECIAL-HANDLING  EQUIPMENT  OR  ITEMS 
REQUIRED 

303  DESIGN  SPECIAL-HANDLING  EQUIPMENT  (ref.  MIL-STD-1365) 

304  DETERMINE  ITEM  PRESERVATION  AND  PACKING  REQUIRE¬ 
MENTS 

305  DETERMINE  ITEM  STORAGE  INSPECTION  REQUIREMENTS 
AND  PROCEDURES 

306  DETERMINE  ITEM  HANDLING  PROCEDURES 

307  DETERMINE  UNIT  PACKING  REQUIREMENPS  AND  QUAN- 
TITY  PER  UNIT  PACKAGE 

308  DETERMLNE  TYPES  AND  QUANTITIES  OF  STANDARD  PHST 
EQUIPMENTS  REQUIRED 

309  DETERMINE  PACKING  AND  UNPACKING  PROCEDURES 

310  DETERMINE  MANPOWER,  SKILLS,  AND  TRAINING  REQUIRE- 
MENTS  RELATED  TO  PHST  FOR  EACH  STORAGE/USAGB  SITE 

31 1  INTEGRATE  PHST  DESIGN  REQUIREMENTS  INTO  ITEM 
DESIGNS  AND  DOCUMENTATION 

312  INTEGRATE  PHST  PROCEDURES  INTO  TECHNICAL  MANU¬ 
ALS  AND  PROCEDURAL  GUIDES 


RELIABILITY  PROGRAM  TASKS  PER  MIL-STD-785 


C  D  F  P  O 

O  it  S  R  & 

N-Y..D-Q.  -S 

T  T  A  A  C 

T  T  A  A  C 

T  T  A  A 

T  A  A  T 

T  A  A  T 
A  A  A  A 


PROGRAM  SURVEILLANCE  AND  CONTROL 

101  RELIABILITY  PROGRAM  PLAN 

102  MONITOR/CONTROL  OF  SUBCONTRACTORS  AND  SUPPLIERS 

103  PROGRAM  REVIEWS 

104  FAILURE  REPORTING,  ANALYSIS,  AND  CORRECTIVE  ACTION 
SYSTEM  (FRACAS) 

105  FAILURE  REVIEW  BOARD 

106  THERMAL  MANAGEMENT  CONTROL  (TMC)  PROGRAM 
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Table  XSH-4.  Support  task  application  matrix  (continued). 


200  DESIGN  AND  EVALUATION 
T  T  C  C  201  RELIABILITY  MODELING 

T  A  C  C  202  RELIABILITY  ALLOCATIONS 

T  T  C  C  203  RELIABILITY  PREDICTIONS 

T  T  C  C  204  FAILURE  MODES,  EFFECTS,  AND  CRITICALITY  ANALYSIS 

(FMECA) 

T  C  205  SNEAK  CIRCUIT  ANALYSIS  (SCA) 

T  C  206  ELECTRONIC  PARTS/CIRCUrrS  TOLERANCE  ANALYSIS 

T  T  A  A  C  207  PARTS  CONTROL/APPLICATION  PROGRAM 

T  T  A  A  A  208  RELIABILITY  CRITICAL  ITEMS 

T  A  A  C  209  EFFECTS  OF  FUNCTIONAL  TESTING,  STORAGE,  HANDLING, 
PACKAGING,  TRANSPORTATION,  AND  MAINTENANCE 
A  A  C  C  210  THERMAL/RELIABILITY  DESIGN  TRADE  STUDIES 

A  A  C  211  THERMAL/RELIABILITY  DESIGN  ANALYSIS 

300  DEVELOPMENT  AND  PRODUCTION  TESTING 
T  A  A  C  301  ENVIRONMENTAL  STRESS  SCREENING  (ESS) 

T  A  302  RELIABILITY  DEVELOPMENT/GROWTH  TESTING 

T  A  A  C  303  RELIABILITY  QUALIFICATION  TEST  (RQT) 

T  A  C  304  PRODUCTION  RELIABILITY  ACCEPTANCE  TEST  (PRAT) 

T  A  C  305  THERMAL  DESIGN  VALIDATION  TEST  (TDVT)  PROGRAM 


SYSTEM  SAFETY  PROGRAM  REQUIREMENTS  PER  MIL-STD-382B 


C  D  F  P  O 

O  &  S  R  & 

N  V  D  O  S 

100  SYSTEM  SAFETY  PROGRAM 
A  A  A  >.  C  101  SYSTEM  SAFETY  PROGRAM  PLAN 

T  T  T  T  T  102  INTEGRATION/MANAGEMENT  OF  SUPPORTING  SUBTSAMS 

T  T  T  T  T  103  SYSTEM  SAFETY  PROGRAM  REVIEWS 

A  A  A  A  C  104  SYSTEM  SAFETY  GROUPAVORKING  GROUP  SUPPORT 

T  A  A  A  A  105  HAZARD  TRACKING  AND  RISK  RESOLUTION 

A  A  A  A  T  106  TEST  AND  EVALUATION  SAFETY 

A  A  A  A  C  107  SYSTEM  SAFETY  PROGRESS  SUMMARY 

T  T  T  T  T  108  QUALIFICATION  OF  SUPPORT  TEAM  SAFETY  ENGINEERS 
AND  MANAGERS 

200  DESIGN  AND  EVALUATION 
ATT  201  PRELIMINARY  HAZARD  LIST 

A  A  A  C  C  202  PRELIMINARY  HAZARD  ANALYSIS 

A  A  C  C  203  SUBSYSTEM  HAZARD  ANALYSIS 

A  A  C  O  204  SYSTEM  HAZARD  ANALYSIS 

T  A  A  C  C  205  OPERATING  AND  SUPPORT  HAZARD  ANALYSIS 

A  A  A  C  C  206  OCCUPATIONAL  HEALTH  HAZARD  ASSESSMENT 

T  A  A  T  C  207  SAFETY  VERIFICATION 

T  T  T  T  205  TRAINING 

T  T  T  T  T  209  SAFETY  ASSESSMENT 

T  T  T  T  T  210  SAFETY  COMPLIANCE  ASSESSMENT 

A  A  A  A  211  SAFETY  REVIEW  OF  ECPs  AND  REQUESTS  FOR  DEVIATIONS 
AND  WAIVERS 
212  (RESERVED) 

T  A  A  A  213  GFE/GFP  SYSTEM  SAFETY  ANALYSIS 


xxiwo 


Table  XXII -4.  Support  task  application  matrix  (continued). 


T  A  A  C  C 
T  A  A  C  C 
T  A  A  C  C 
T  A  A  C  C 
T  A  A  C  C 
T  A  A  C  C 
T  A  A  C  C 


SOFTWARE  HAZARD  ANALYSIS 

301  SOFTWARE  REQUIREMENTS  HAZARD  ANALYSIS 

302  TOP-LEVEL  DESIGN  HAZARD  ANALYSIS 

303  DETAILED  DESIGN  HAZARD  ANALYSIS 

304  CODE-LEVEL  SOFTWARE  HAZARD  ANALYSIS 

305  SOFTWARE  SAFETY  TESTING 

306  SOFTWARE/USER  INTERFACE  ANALYSIS 

307  SOFTWARE  CHANGE  HAZARD  ANALYSIS 


TESTABILITY  PROGRAM  PER  MIL-STD-2165 


C  D  F  P  O 

O  &  S  R  & 

N  V.  .D  .  .Q  S 

A  A  A  T  C 

T  A  A 


PROGRAM  MONITORING  AND  CONTROL 

101  TESTABILITY  PROGRAM  PLANNING 

102  TESTABILITY  REVIEWS 

103  TESTABILITY  DATA  COLLECTION  AND  ANALYSIS  PLANNING 


AAA 

TAT 

TAT 


200  DESIGN  AND  ANALYSIS 

201  TESTABILITY  REQUIREMENTS 

202  TESTABILITY  PRELIMINARY  DESIGN  AND  ANALYSIS 

203  TESTABILITY  DETAIL  DESIGN  AND  ANALYSIS 


TAT 


300  TEST  AND  EVALUATION 

301  TESTABILITY  INPUTS  TO  MAINTAINABILITY  DEMONSTRA¬ 
TION 


PRODUCT  ASSURANCE  PER  DOD-STD-2107  (Partial) 


C  D  F  P  O 

O  &  S  R  & 

N  V  D  O  S 

R  R  R  4.1.1 

AAA  4.1.2 

A  A  A  A  4.1.3 

R  R  R  4. 1.4.1 

T  R  4. 1.4.2 

R  R  R  4. 1.5.1 

R  R  R  4.1.5.2 

R  R  C  4.1.6 

R  R  R  4.17 

C  A  A  4.1.8 


A  A  A  A  R  4.1.9 


GENERAL 

MANAGEMENT  POLICY 
PROGRAM  PLANNING 
EDUCATION  AND  TRAINING 
CERTIFICATION  OF  PERSONNEL 
AUDIT  CONDUCT 

AUDIT  REPORT  AND  CORRECTIVE  ACTION 
INTEGRATED  DATA  SYSTEM 
INTEGRATED  TEST  PROGRAM 

PROBLEM/FAILURE  REPORTING.  ANALYSIS.  AND  CORRECTIVE 
ACTION 

4.1. 8.1  PROBLEM/FAILURE  REPORTING 

4. 1.8.2  PROBLEM/FAILURE  INVESTIGATION  AND  ANALYSIS 

4. 1.8.3  CORRECTIVE  ACTION 
CONFIGURATION  MANAGEMENT  (see  MIL-STD-480) 

4.1.9.1  CONFIGURATION  IDENTIFICATION 

4.1.9.2  CONFIGURATION  CONTROL 

4.1.93  CONFIGURATION  CONTROL  BOARDS 

4. 1.9.4  CONFIGURATION  STATUS  ACCOUNTING 

4.1.9.5  CONFIGURATION  AUDITS 
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Table  XXlI-4.  Support  task  application  matrix  (continued). 


T  A  C  4.1.10  NONCONFORMING  MATERIAL  CONTROL  (MIL-STD-1520  ALTER¬ 
NATE) 

4.1.10.1  IDENTIFICATION  AND  SEGREGATION 

4.1.10.2  MISSED  OPERATIONS 

4. 1. 10.3  PRELIMINARY  REVIEW 

4.1.10.4  MATERIAL  REVIEW  BOARD 

4.1.10.5  NONCONFORMANCE  REQUESTS  FOR  DISPOSITION 

4. 1 . 10.6  SUBCONTRACTOR  MATERIAL  REVIEW  BOARD 
A  A  C  4.1.11  SAMPLING  PLAN' 

A  A  4.1.12  QUALITY  COST  DATA 
T  T  T  T  C  4.1.13  SOFTWARE 

T  A  A  C  4.1.14  TECHNICAL  DATA  QUALITY  ASSURANCE 


T  A  A  T  4.2 

T  A  A  T  4.2.1 

T  A  A  T  4.2.2 

A  A  4.2.3 
T  T  T  4.2.4 

A  A  A  A  C  4.2.5 

AAA  4.2.6 
A  A  A  C  4.2.7 

T  T  T  4.2  8 

A  A  A  A  4.2.9 


SUPPLIER  QUALITY  ASSURANCE  PROGRAM  REQUIRE¬ 
MENTS  (M1L-STD.1535  ALTERNATIVE) 

GENERAL 

SELECTION  OF  PROCUREMENT  SOURCE 
APPROVED  SOURCE  LIST 
SURVEYS  OF  SUBCONTRACTORS  OPERATIONS 
PROCUREMENT  DOCUMENT  PROVISIONS 
PROCUREMENT  DOCUMENT  REVIEW 
PROCUREMENT  DOCUMENT  CHANGE  CONTROL 
CONTRACTOR/SUBCONTRACTOR  COORDINATION  AND  CORREC¬ 
TIVE  ACTION 

CONTRACTOR  SOURCE  SELECTION 


A  A  A  C 
AAA 
T  A  A  C  C 


T  A  A  C 
T  A  C  C 
T  T  T  C  C 

A  A  A  A 

T  T  T 

T  T  C 

R  R  R 

R  R 
T  R 
T  R 
T  T 
R  T 

R  R  R  T  C 


4.3  DEVELOPMENT 

4.3.1  MISSION/SYSTEM  ANALYSIS 

4.3.2  DESIGN  ANALYSIS 

4.3.2. 1  PARAMETER  STUDIES 

4 .3.2.2  CLASSIFICATION  OF  CHARACTERISTICS  (see  DOD-STD- 
2101) 

4.3.23  SNEAK  ANALYSIS 

4 .3.2.4  FAILURE  MODE,  EFFECT,  AND  CRITICALITY  ANALYSIS 

4.3.2.5  STRESS  ANALYSIS  OF  PARTS  AND  MATERIALS 

4.3.2. 6  WORST-CASE  ANALYSIS 

4  3.2.7  MAINTAINABILITY  AND  RELIABILITY  ANALYSIS 
4.3..I.8  LOGISTICS  ANALYSIS 
4.3.2.9  PRODUCIBILITY  ANALYSIS 

4.3.3  DESIGN  PRACTICES  AND  DOCUMENTATION 

4.3.4  PARTS  AND  MATERIALS  SELECTION  &  IDENTIFICATION 

4.3.5  HUMAN  ENGINEERING  PROGRAM 

4.3.6  DESIGN  REVIEW 

4.3.7  KEY  COMPONENTS 

4.3.8  CONTROL  OF  KEY  COMPONENTS 

4.3.10  DEVELOPMENT  TESTS 

4.3.10.1  DESIGN  TESTS 

4.3.10.2  ENGINEERING  EVALUATION  TESTS 
4.3  10.5  QUALIFICATION  TESTS 

4.3.10.6  RELIABILITY  DEMONSTRATION  TESTS 

4.3.10.7  MAINTAINABILITY  DEMONSTRATION  TESTS 

4.3. 1 1  RELIABILITY  AND  MAINTAINABILITY  ACCOUNTING 

4.3.11.1  RELIABILITY  MODELING 

4  3.11.2  RELIABILITY  PREDICTION 

4.3.11.3  RELIABILITY  APPORTIONMENT 

4.3.11.4  RELIABILITY  EVALUATION 

4.3.1 1.5  MAINTAINABILITY  MODELING 

4.3.11.6  MAINTAINABILITY  PREDICTION 

4.3. 1 1.7  MAINTAINABILITY  APPORTIONMENT 
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Table  XXII -4.  Support  task  application  matrix  (continued). 


R  R  R  T 


T  A 
T  A 
A  C 


4.3.11.8  MAINTAINABILITY  EVALUATION 

4.3.11.9  AVAILABILITY  EVALUATION 

4.3.12  TEST  AND  INSPECTION  DU  RING  DEVELOPMENT 

4.3.12.1  DEMONSTRATION  AND  VALIDATION 

4.3.12.2  FULL-SCALE  DEVELOPMENT 

4.3.13  MATERIAL  IDENTIFICATION  l  ND  HANDLING 

4.3.14  MANUFACTURING  CONTROL  DURING  DEVELOPMENT 

4.3.15  READINESS  REVIEW  FOR  PRODUCTION  OPERATIONS 


T  A  C  4.4  PRODUCTION 

4.4.1  GENERAL 

4.4.2.1  PARTS  AND  MATERIALS  CONTROL 

4.4.2.2  PROCESS  CONTROLS 

4.4.2  3  ASSEMBLY  OPERATIONS 

4.4.2.4  ENVIRONMENTAL  AND  CLEANLINESS  CONTROL 

4.4.2.5  WORKMANSHIP  STANDARD 

4.4.3  TEST  AND  INSPECTION  PLANNING 

4.4.4  QUALITY  VERIFICATION 

4.4.4.1  RECEIVING  TEST  AND  INSPECTION 

4.4.4.2  IN-PROCESS  TEST  AND  INSPECTION 

4.4.4.3  NONDESTRUCTIVE  TESTING  PROCESS 

4.4.4.4  CONFIGURATION  VERIFICATION 

4.4.4.5  ACCEPTANCE  TEST  AND  INSPECTION 
4.4.4.6A  FIRST  ARTICLE  INSPECTION 
4.4.4.6B  QUALIFICATION  TESTS 

4.4.4.6C  PERIODIC  PRODUCTION  TESTS 

4.4.4.7  INSPECTION  INDICATIONS 

4.4.4.8  TEST  AND  INSPECTION  RECORDS 

4.4.5. 1  MATERIAL  PROTECTION 

4.4.5.2  SHIPPING  INSPECTION 

A  4.4.6  DOCUMENTATION  CONTROL  DURING  PRODUCTION 


T  A  A  C 

T  T  T 
T  T  T 
T  R  R 
T  R  R 
T  R  R 
T  T  T 
T  T  T 
T  T  T 
T  R  R 
R  C 


T  T 

R  C 
R  R 
T  T 


4.5  TEST  AND  INSPECTION  EQUIPMENT  AND  STANDARDS 

4.5.1  CALIBRATION  SYSTEM  (see  DoD-STD-45662) 

4.5.1. 1  CALIBRATION  PROCEDURE  PRECEDENCE 

4.5.1.2  INITIAL  INTERVALS  OF  CALIBRATION 

4.5. 1.3  IDENTIFICATION  OF  STANDARDS 

4 .5.1.4  SEALING 

4.5.1.5  MEASUREMENT  AND  TEST  EQUIPMENT  CONTROL 

4.5. 1.6. 1  QUALITATIVE  DATA 

4 .5. 1.6.2  QUANTITATIVE  DATA 

4.5.1.7  ALLOWABLE  ERROR  OF  STANDARDS 

4 .5.1.8  MEASURING  DEVICES 

4.5.2  TEST  AND  INSPECTION  EQUIPMENT  DESIGN  AND  EVALUATION 

4.5.2. 1  DESIGN 

4.5.2.2  EVALUATION 

4.5.3  TEST  AND  INSPECTION  STATION  OPERATIONAL  PROOFING  AND 
CORRELATION 

4 .5 .3.1  OPERATIONAL  PROOFING 

4.5S.2  TEST  AND  INSPECTION  STATION  LOGS 

4.5.3S  TEST  AND  INSPECTION  STATION  CORRELATION 
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Table  XXII -4.  Support  task  application  natrix  (continued). 
CALIBRATION  REQUIREMENTS  PER  DOD-STD -45662  (PER  MIL-STD-2107) 


c 

D 

F 

P 

0 

0 

& 

S 

R 

& 

tL 

JL 

JL 

JL. 

£ 

R 

R 

R 

4.1 

GENERAL 

4.2 

QUALITY  ASSURANCE  PROVISIONS 

T 

R 

R 

5.1 

CALIBRATION  SYSTEM  REQUIREMENTS 

T 

R 

R 

5.2 

ADEQUACY  OF  STANDARDS 

T 

R 

R 

53 

ENVIRONMENTAL  CONTROLS 

T 

R 

R 

5.4 

INTERVALS  OF  CALIBRATION 

T 

R 

R 

5.5 

CALIBRATION  PROCEDURES 

T 

R 

R 

5  6.1 

EVALUATION  OF  SUSPECT  PRODUCT 

T 

R 

R 

5.6.2 

EVALUATION  OF  CALIBRATION  SYSTEM  ACCURACY 

R 

R 

R 

5.7.1 

DOMESTIC  CONTRACTS  (CALIBRATION  SOURCES) 

T 

T 

T 

5.7.2 

FOREIGN  CONTRACTS  (CALIBRATION  SOURCES) 

T 

R 

k 

5.8 

APPLICATION  AND  REPORTS 

T 

R 

R 

5.9 

CALIBRATION  STATUS 

R 

R 

R 

5.10 

CONTROL  OF  SUBCONTRACTOR  CALIBRATION 

R 

R 

R 

5.11 

STORAGE  AND  HANDLING 

R 

R 

R 

5.12 

AMENDMENTS  AND  REVISIONS 

HUMAN  FACTORS  ENGINEERING  PER  MIL-H-46855B 


c 

D 

F 

p 

0 

0 

& 

S 

R 

& 

tL 

JL 

JL 

JL. 

.£ 

A 

A 

A 

A 

GENERAL  PROVISIONS 

A 

C 

C 

C 

3.1.1.A 

ANALYSIS 

T 

A 

A 

C 

r  l.l.B 

DESIGN  AND  DEVELOPMENT 

0 

A 

A 

C 

b.l.l.C 

TEST  AND  EVALUATION 

T 

T 

A 

C 

3.1.2 

HE  PROGRAM  PLANNING 

A 

A 

A 

3.1.3 

NONDUPLICATION 

A 

A 

A 

3.2 

DETAIL  REQUIREMENTS 

A 

A 

A 

C 

3.2.1 

ANALYSIS 

A 

A 

0 

3.2.1. 1 

DEFINING  AND  ALLOCATING 

A 

A 

0 

3.2.1. 1.1 

INFORMATION  FLOW  AND  PROCESSING  ANALYSIS 

T 

A 

0 

3.2.1. 1.2 

ESTIMATES  OF  POTENTIAL  OPERATOR/MAINTAINER  PROCESS¬ 
ING  CAPABILITIES 

A 

A 

0 

3.2. 1.1.3 

ALLOCATION  OF  FUNCTIONS 

T 

A 

0 

3.2. 1.2 

EQUIPMENT  SELECTION 

A 

A 

A 

C 

3.2. 1.3 

ANALYSIS  OF  TASKS 

T 

A 

A 

C 

3.2. 1.3.1 

GROSS  ANALYSIS  OF  TASKS 

0 

A 

A 

C 

3.2. 1.3.2 

ANALYSIS  OF  CRITICAL  TASKS 

A 

A 

A 

C 

3.2. 1.3.3 

WORKLOAD  ANALYSIS 

T 

T 

A 

c 

3.2. 1.3.4 

CONCURRENCE  AND  AVAILABILITY 

T 

T 

A 

c 

3.2. 1.4 

PRELIMINARY  SYSTEM  DESIGN 

T 

A 

A 

c 

3.2.2 

HE  IN  EQUIPMENT  DETAIL  DESIGN 

A 

A 

A 

3.2.2. 1 

STUDIES,  EXPERIMENTS,  AND  LABORATORY  TESTS 

3.2.2. 1. 1 MOCKUPS  AND  MODELS 

3.2.2. 1.2  DYNAMIC  SIMULATION 

0 

A 

A 

c 

3.2.2.2 

EQUIPMENT  DETAIL  DESIGN  DRAWINGS 

T 

A 

A 

c 

3.2.2J 

WORK  ENVIRONMENT,  CREW  STATIONS,  AND  FACILITIES 

DESIGN 
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Table  XXII -4.  Support  task  application  matrix  (continued). 

0  A  A  C  3  2.2.4  HE  IN  PERFORMANCE  AND  DESIGN  SPECIFICATIONS 
0  A  A  C  3.2.2.5  EQUIPMENT  PROCEDURE  DEVELOPMENT 

T  T  A  8.2.3  HEINT&E 

3.2.3. 1  PLANNING 

3. 2.3.2  IMPLEMENTATION 

3.2.3.3  FAILURE  ANALYSIS 

A  A  A  A  3.2.4  COGNIZANCE  AND  COORDINATION 
A  A  A  A  3.3  DATA  REQUIREMENTS 

T  T  A  A  3.3.1  TRACEABILITY 

A  A  A  A  3.3.2  ACCESS 

A  A  A  A  3.4  DRAWING  APPROVAL 

CODES:  A -APPLIES 

C  -  CONDITIONALLY  APPLIES,  ESPECIALLY  FOR  CHANGES 
O  -  OPTIONAL  AS  PART  OF  OVERALL  PROGRAM  TAILOR¬ 
ING 

R  -  HIGHLY  RECOMMENDED 
T  -  TAILORED  REQUIREMENTS 


DEALING  WITH  CONTRACTORS 


The  Project  Manager  (PM)  and  Systems  Engineer  (SE)  must  both  work  with 
contractors,  especially  when  significant  portions  of  the  project  tasks  are  being  accom¬ 
plished  under  contract.  In  this  relationship,  the  PM/SE  must  strike  a  balance 
between  competing  elements  without  compromise.  These  elements  include: 

The  Public  Interest  (long  term)  The  Public  Interest  (short  term) 

Being  a  good  customer  Ethics 

Maintaining  team  spirit  Objectivity 

Responsibility  Accountability 

These  are  not  mutually  exclusive  elements,  but  the  unwary  PM/SE  can  fall  into  seri¬ 
ous  traps  if  the  right  balance  is  not  maintained. 


BEING  A  GOOD  CUSTOMER 


It  is  important  to  be  a  good  customer,  but  this  is  made  difficult  by  the 
assumed  adversarial  relationships  inherent  in  many  contractual  procedures  and  by 
the  requirements  to  maintain  an  ethical  profile  avoiding  even  the  appearance  of  any 
impropriety.  Still,  it  is  possible.  The  key  to  being  a  good  customer  is  OPEN  BUT 
PROPER  COMMUNICATIONS.  The  first  of  these  communications  are  the 
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statements  of  requirements  in  the  statement  of  work,  specifications,  contract  data 
requirements,  and  contract  clauses.  C.rte  the  requirements: 

clearly  and  concisely 

completely 

to  be  tailored  to  obtain  all  that  is  required  without  overspecification 

to  minimize  change,  but  identifying  known  changes  and  making  provi¬ 
sions  for  necessary  changes  in  an  orderly  manner 

In  the  statement  of  work,  delineate  the  responsibilities  between  the  Government  and 
the  contractor,  and  state  the  provisions  for  open  but  proper  communications.  In  the 
CDRL,  obtain  all  the  INFORMATION  that  is  needed  while  minimizing  data.  In  the 
contract,  hold  the  contractor  accountable  for  performance,  cost,  and  schedule  with 
default  provisions  having  teeth  (but  also  allowing  for  agreed-upon  changes). 

Too  often,  contracts  are  established  without  clearly  identifying  requirements, 
this  practice  exposes  the  Government  to  unnecessary  costs  and  puts  the  contractor  in 
the  uncomfortable  position  of  being  unable  to  perform  the  tasks  satisfactorily  and 
professionally.  This  is  the  sort  of  thing  that  embarrassing  news  stories  are  made  of 

Remember  -  VERBAL  AGREEEMENTS  AREN’T  WORTH  TIIE  PAPER  THEY'RE 
PRINTED  ON  (but  they  can  cost  much  more). 

Remember  —  a  contract  places  requirements  on  both  parties;  make  sure  that  the 
Government  lives  up  to  its  part  in  the  bargain  by  providing  GFE/GFM  on  time  and 
that  meets  specifications,  by  providing  timely  reviews  and  approvals,  and  by  making 
prompt  payments  (which  are  often  held  up  by  the  failure  of  a  PM  or  SE  to  sign  the 
receipt  forms  on  time). 

Remember  —  “Do  not  muzzle  the  ox”;  contractors  art  entitled  to  fair  and  equitable 
compensation  for  their  efforts. 

Remember  —  contractors  are  people.  They  deserve  to  be  treated  professionally  and 
with  respect.  (Likewise,  the  PM/SE  should  act  in  a  way  to  command  the  respect  of 
the  contractors.)  Treat  contractors  as  part  of  the  team  as  people,  but  remember  that 
the  contractor  organization  exists  to  make  a  profit  at  your  project’s  expense.  Ensure 
that  it  is  money  well  spent  and  that  the  project  obtains  what  is  required  by  the  time 
the  contract  is  finished. 


OPEN  BUT  PROPER  COMMUNICATION 


Good  project  communications  must  include  the  contractor(s)  participating  on 
the  project  team.  These  communications  must  be 


timely, 

accurate, 
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complete, 

but  within  established  contractual  channels. 

All  verbal  communications  should  be  strictly  unofficial  and  backed  up  by  official  writ¬ 
ten  communications  or  records.  Reviews  and  meetings  are  an  excellent  forum  for  dis¬ 
cussing  issues  and  reaching  conclusions  on  required  actions,  but  a  formal  written 
record  should  be  maintained.  Telephone  conversations  should  also  be  documented  in 
telephone  conversation  records  and  filed  appropriately.  All  decisions  and  communica¬ 
tions  which  affect  the  contractual  requirements  (including  SOW  and  specification 
requirements)  must  be  communicated  and  approved  through  the  contracting  officers. 
It  should  be  made  contractually  impossible  for  verbal  direction  to  change  the  con- 
t-n'-ted  requirements.  Proper  communications  protect  sensitive  information,  includ¬ 
ing 


security  information, 
privacy  information, 
and  proprietary  information. 
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XXIII.  OTHER  MANAGEMENT  TASKS 


EMX  MANAGEMENT 


One  of  the  most  severe  environments  encountered  by  equipments  in  military 
applications  is  the  electromagnetic  (EM)  environment.  The  severity  of  military  EM 
environments  tends  to  grossly  exceed  that  of  otherwise  comparable  commercial  EM 
environments;  therefore,  an  effective  program  of  EM  management,  assessment,  and 
engineering  is  required  to  assure  system  effectiveness.  This  need  extends  equally  to 
off-shelf  equipments  and  to  equipment  developments,  although  the  actions  taken  are 
affected  by  the  equipment  source. 

Problems  arising  from  EM  sources  are  of  three  types: 

•  Unwanted  emissions  from  the  equipment 

•  Damage  to  the  equipment  by  the  EM  environment 

•  Undesirable  modification  to  equipment  performance  due  to  emissions  from 
other  equipments 

Tho  damaging  effects  of  these  problems  fall  into  one  or  more  of  the  following 
categories: 

•  Hazards  to  personnel 

•  Hazards  to  ordnance 

•  Hazards  to  other  equipments 

•  Degraded  equipment  performance 

o  Degraded  security 

The  engineering  tools  which  are  available  to  combat  EM  problems  include: 

•  Filtering 

•  Shielding 

•  Grounding 

•  Application  of  special  technology 

•  Careful  component  selection  and  application 

«  Installation  control 

Each  of  these  tools  costs  money  to  implement;  how  much  money  depends  on 
the  degree  of  protection  needed  and  on  the  point  at  which  protective  measures  are 
applied  in  the  design  of  the  equipment.  When  design  changes  are  mandated,  they  are 


XXIII-1 


always  more  expensive  than  proper  initial  design;  furthermore,  design  changes  are 
always  more  extensive  as  the  equipment  design  matures.  The  variety  and  impact  of 
the  problems  and  prospective  solutions  demand  management  attention  in  order  to 
effect  an  acceptable  product  and  to  efficiently  utilize  project  resources. 

There  are  a  number  of  terms  which  are  somewhat  peculiar  to  the  EM  field: 

EMC  Electromagnetic  compatibility  —  a  term  encompassing  the 

issues  of  an  equipment  operating  properly  within  its  EM 
environment  and  not  causing  interference  to  other  equipments 

EMI  Electromagnetic  interference  —  negative  effects  of  one 

equipment  on  another 

EMP  Electromagnetic  pulse  —  an  EM  effect  of  a  nuclear  blast 

EMX  A  blanket  term  covering  all  EM-related  problems 

HERO  Hazardous  electromagnetic  radiation  to  ordnance  (see 
MIL-STD-1377) 

RADHAZ  Electromagnetic  radiation  hazard  to  personnel 

TEMPEST  A  term  covering  security  problems  which  are  EM  design 
oriented 

This  list  is  not  complete  but  is  sufficient  for  the  discussion  which  follows. 

Hazards  to  personnel  include  shock  hazards  and  RADHAZ.  The  approaches 
toward  reducing  these  hazards  include  safety  interlocks,  protective  enclosures  around 
high-hazard  sections,  good  grounding  techniques,  and  warning  labels  and  signs. 
Requirement  1  of  MIL-STD-454  covers  these  approaches.  No  equipment  or  system 
should  be  exempted  from  these  provisions. 

Hazards  to  ordnance  are  normally  treated  by  control  of  the  ordnance  design 
and  by  ordnance  handling  procedures  which  require  shutting  down  all  emitters  in  the 
2-32-MHz  band  (the  frequencies  of  greatest  ordnance  susceptibility).  The  equipments 
which  may  be  connected  to  ordnance,  directly  or  indirectly,  must  be  designed  to  pro¬ 
tect  sensitive  circuits  (particularly  firing  circuits)  from  the  EM  environment.  The 
steps  required  include  proper  grounding  techniques,  shielding,  filtering  against  tran¬ 
sient  and  spurious  signals,  and  designs  to  preclude  false  signaling,  safety  interlocks 
and  fail-safe  designs  should  be  integral  to  all  equipments  connected  to  ordnance 

If  an  equipment  will  be  operating  remotely  and  independently  from  any  other 
equipment,  EMI/EMC  is  not  a  particular  problem;  however,  most  applications  will 
require  shared  power  systems,  antenna  systems,  control  systems,  or  installation 
spaces.  Usually,  EMC  will  bo  important.  Figure  V-3  shows  the  relationship  between 
the  significant  EMI  and  EMC  specifications  and  standards.  “Suggestions  for  Design¬ 
ers  of  Navy  Electronic  Equipment"  (NOSC  TD  250)  contains  many  tips  on  good  EM 
engineering  techniques.  The  project  manager  should  ensure  that  lead  engineers  are 
familiar  with  EM  design  practices  and  that  the  test  program  incorporates  EM  test¬ 
ing.  MIL-STD-461  is  the  primary  EMI  requirements  standard.  It  describes  the  “nor¬ 
mal  worst  case”  EM  environment,  as  such,  it  overspecifies  some  applications  and 
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underspecifies  others.  Where  more  precise  validated  information  is  available  for  a 
particular  application,  it  should  be  used  in  lieu  of  MIL-STD-461.  For  general  applica¬ 
tions  or  other  applications  in  which  a  large  number  of  equipments  are  collocated  and 
are  subject  to  changes  in  configuration,  then  apply  the  more  stringent  requirements. 
The  test  requirements  of  MIL-STD-462  are  valid  even  if  the  test  parameters  are 
modified  by  better  information.  However,  the  broadband  tests  of  CE01  and  the 
requirements  of  CE05  are  difficult,  if  not  impossible,  to  test  in  a  valid  manner;  they 
should  be  excluded  from  or  modified  for  project  test  requirements.  Particular  atten¬ 
tion  should  be  paid  to  spurious  emissions. 

OfT-shelf  equipments  should  be  tested  during  the  screening  process  to  assure 
compliance  to  EM  requirements,  because  of  the  peculiarly  stringent  military  EM 
environments,  many  ofT-shelf  commercial  equipments  will  require  filters  and  careful 
installation  control  to  adequate  compatibility.  Some  commercial  equipments  will 
show  susceptibility  to  damage  in  military  EM  environments  and  should  be  disquali¬ 
fied  if  the  damaging  conditions  occur  during  normal  operations  or  frequently  and  if 
adequate  protection  .annot  be  added  economically. 

In  the  conduct  of  an  EMC  program,  MIL-HDBK-235,  MIL-HDBK-237,  and 
MIL-HDBK-238  should  prove  useful.  MIL-HDBK-237  discusses  EMC  program  man¬ 
agement. 

MIL-HDBK-235  describes  specific  EM  environments.  MIL-HDBK-238 
describes  numerous  radiation  hazards  and  ways  to  handle  them.  Also,  MIL- 
HDBK-24I  provides  guidance  for  designing  power  supplies  and  filters  for  EMC. 

EMP  is  a  very  difficult  problem  and  an  expensive  one  to  solve.  Because  it  is  so 
expensive  to  treat,  EMP  management  should  be  applied  only  to  vital  systems.  Con¬ 
sider  all  systems  wiped  out  by  EMP  damage  except  your  system  and  EMP  qualified 
systems;  if  your  system  is  still  essontial  to  critical  operational  missions,  then  it  is  a 
candidate  for  EMP.  All  the  EMX  tools  must  be  brought  into  play  to  attack  EMP; 
therefore,  it  is  essential  that  design  personnel  be  familiar  with  the  EMP  problem  and 
approaches  to  its  solution.  Courses  are  available  on  EMP  which  should  be  attended 
by  all  designers,  bake  a  common  sense  approach  to  the  problem.  Must  the  system 
operate  flawlessly  during  the  EMP,  or  can  some  downtime  be  tolerated?  Specially 
designed  microcircuits  and  semiconductors  may  be  required  to  survive  EMP.  EMP  is 
characterized  by  extremely  fast  rise  times  and  very  broadband  high-intensity  fields; 
both  effects  must  be  considered.  Fast-acting  protective  devices  may  be  indicated.  An¬ 
other  approach  is  to  design  the  equipment  so  that  EMP  damage  will  be  confined  to  a 
known  component  or  module,  then  a  spare  can  be  designed  into  the  system,  hut  out  of 
the  circuit  to  provide  a  low  downtime.  Include  EMP  testing  in  your  program.  Knowl¬ 
edgeable  personnel  and  early  design  attention  aro  essential  to  efficient  EMP  manage¬ 
ment. 


TEMPEST  is  similar  to  EMI  in  its  basic  manifestations,  causes,  and  cures. 
However,  TEMPEST  is  concerned  with  signal  intelligence  whereas  EMI  is  not;  this  is 
reflocted  in  the  respective  testing  philosophy.  The  power  and  frequency  ranges  at 
which  intelligence  can  be  recovered  are,  in  general,  more  stringent  than  those  which 
cause  interference.  TEMPEST  requires  early  consideration  of  knowledgeable  design 
practice.  The  standards  for  TEMPEST  are  NACSIM  5100A  and  NACSEM  5103.  Be¬ 
cause  it  is  a  specialized  engineering  discipline,  project  managers  for  equipment  which 
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handles  classiSed  information  should  request  guidance,  technical  assistance,  and  pub¬ 
lications  from  the  Naval  Electronic  Systems  Security  Engineering  Center  (NESSEC), 
3801  Nebraska  Avenue  NW,  Washington,  DC  20390.  Commands  and  activities  of  the 
Navy  should  send  requests  directly  to  the  Commanding  Officer,  NESSEC.  Industrial 
organizations  performing  on  contract  should  submit  requests  though  the  contracting 
officer  and  the  sponsoring  activity.  Other  naval  commands  and  activities,  and  non- 
Navy  agencies,  may  contact  NESSEC  for  similar  assistance.  It  is  essential  that  this 
contact  be  established  early  to  assure  adequate  planning,  funding,  and  execution  of 
TEMPEST-related  tasks. 

When  installation  control  is  to  be  employed,  as  it  musi  be  in  secure  systems 
and  EMP-protected  systems,  the  control  requirements  must  be  cited  in  the  installa¬ 
tion  planning  documentation  (see  chapter  XVIII).  Special  installation  controls  will 
involve  additional  design,  planning,  and  supervision  by  the  installing  activity.  Wher¬ 
ever  possible,  it  is  advisable  to  build  features  into  the  equipment  which  force  proper 
installation  to  avoid  installation  control  problems;  however,  systems  with  intercon¬ 
necting  cabling  to  units  in  remote  locations  must  contend  with  the  problems  of  instal¬ 
lation  control. 

Standards  for  bonding  and  grounding  techniques  are  found  in  MIL-STD-1310 
and  MIL-B-5087. 


PARTS  MANAGEMENT 

The  management  of  parts  selection  and  application  i3  essential  to  the  success 
of  any  engineering  project.  Unlike  other  management  tasks,  parts  management  is  not 
subject  to  the  rigorously  well-defined  guidance  that  are  other  tasks,  such  as  configu¬ 
ration  management. 

Parts  management  policy  must  be  set  by  the  project  manager  to  conform  to 
the  conditions  which  exist  for  that  project;  the  policy  is  implemented  by  the  design¬ 
ers.  In  this  context,  a  part  must  be  considered  any  piece  part,  module,  subassembly, 
assembly,  or  unit  which  might  be  stocked  as  an  item  of  supply  support  or  the  compo¬ 
nents  making  up  such  an  item.  Obviously  the  parts  management  policy  must  eventu¬ 
ally  dictato  the  provisioning  decisions  for  an  equipment,  since  the  policy  dictates  the 
design  which  must  be  provisioned.  Past  attempts  to  mandate  a  general  parts  manage¬ 
ment  policy  have  amounted  to  a  blanket  standard  piece  parts/standard  module  policy 
which  has  not  always  been  consistent  with  individual  project  goals,  i.e.,  utilization  of 
commercial  equipment,  specialized  technologies,  or  low-cost/high-availability  parts  or 
attainment  of  reliability  goals.  A  much  broader  policy  of  standardization  is  enunci¬ 
ated  by  NAVMATINST  4120.97  (series)  which  is  summarized  in  chapter  XIV  in  the 
section  titled  Standardization  of  Components/Equipment  (C/E).  The  Low  Cost  Elec¬ 
tronics  study  shows  that  the  provisions  of  this  instruction  are  largely  ignored  or 
inappropriately  applied  because  of  a  lack  of  good  technical  guidance  to  project/acqui¬ 
sition  managers.  This  section  will  attempt  to  provide  such  guidance. 

In  general,  five  factors  must  be  considered  in  setting  a  parts  management  pol¬ 
icy: 

Performance 

Reliability 
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Provisioning 

Cost 

In  addition,  the  project  manager  must  consider  parts  availability,  since  delays  in 
parts  deliveries  may  have  a  very  negative  effect  on  the  project  schedule.  These  fac¬ 
tors,  as  they  interact  with  each  other,  may  be  lumped  into  three  categories  of  issues; 

Standardization 

Effectiveness 

Efficiency 

STANDARDIZATION 

There  are  two  types  of  standardization— intrasystem  and  intersystem.  The 
goals  of  intrasystem  standardization  are  to: 

«  Increase  the  producibility  of  the  system  by  reducing  the  number  of  differ¬ 
ent  kinds  of  components. 

•  Increase  system  supportability  through  widespread  intrasystem  com¬ 
monality  of  designs,  modules,  etc. 

•  Decrease  system  documentation  costs  through  commonality  of  designs, 
modules,  etc. 

•  Create  a  situation  in  which  all  items  provisioned  can  be  ordered  in  eco¬ 
nomically  large  quantities  (see  fig  VI-1). 

•  Decrease  system  downtime  for  parts  (see  fig  IV-4). 

•  Decrease  system  design  time  by  limiting  the  choices  available  to  the 
designer. 

•  Establish  standard  intrasystem  interfaces. 

Most  simply,  intrasystem  standardization  strives  to  make  the  widest  possible  use  of 
the  fewest  possible  different  kinds  of  parts  and  designs.  The  greatest  advantages  of 
such  programs  as  the  Standard  Electronics  Module  (SEM)  and  the  Standard  Hard¬ 
ware  Program  (SHP)  lie  in  the  ready  availability  through  them  of  design  building 
blocks  for  application  in  intr^system  standardization  efforts.  The  goals  of  intersystem 
standardization  are  to  minimize  the  number  of  different  logistics  items  which  must 
be  supported  and  to  maximize  interchangeability  between  items  of  like  functions. 

The  following  steps  are  recommended  to  implement  the  standardization  objec¬ 
tives  of  parts  management: 

1.  Establish  a  project  policy  of  maximizing  intrasystem  standardization. 

2.  Wherever  possible,  make  use  of  existing  standards  (standard  interfaces, 
standard  modules,  standard  parts,  standard  equipments,  standard  test 
provisions,  etc.);  howevor,  do  not  enforce  this  provision  to  these  degrada¬ 
tion  of  other  project  requirements.  Established  industry  standards  should 
be  considered  as  well  as  military  standards. 
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3.  Minimize  system-peculiar  designs  and  components. 

4.  In  selecting  parts,  choose  preferred  parts  (MIL-STD-242)  over  other  stan¬ 
dard  parts  (controlled  by  military  specification),  standard  parts  over  con¬ 
trolled  parts  (nonstandard  parts  already  supported  in  the  National  Stock 
System),  and  controlled  parts  over  all  other  parts.  This  step  is  imple¬ 
mented  in  accordance  with  MHrSTD-965  and  MIL-STD-143. 

5.  Where  other  factors  (such  as  cost  or  availability)  militate  against  the  use 
of  the  part  which  would  otherwise  be  selected  under  step  4,  select  a  part 
which  can  be  replaced  by  the  step  4  selection  for  repair/provisioning  pur¬ 
poses  and  show  the  step  4  selection  in  the  provisioning  documentation. 
This  step  is  implemented  in  accordance  with  Requirement  7  of  MIL- 
STD-454. 

6.  Document  all  system  interfaces,  down  to  the  level  of  standardization,  using 
functional  specifications  as  discussed  in  chapter  VI. 

Step  6  is  particularly  important,  since  it  establishes  the  mechanism  for  future  design 
evolution  within  the  framework  of  standardization:  figure  XXIII-1  illustrates  this 
mechanism. 


(A) 

SPEC 

xxx 


(81 


(C) 

SPEC  00 
XXX  A 


(D) 
SPEC 
XXX  B 


SPEC  YYY 


SYSTEM  A  SYSTEMS  A.  B.  C  SYSTEM  E 

SYSTEM  B  SYSTEM  D  SYSTEM  F 

SYSTEM  C  SYSTEM  G 


(A)  ITEM  A  IS  DEVELOPED  FOR  SYSTEM  A  AND  DOCUMENTED  BY  SPEC  XXX. 

(B)  ADDITIONAL  USE  FOR  ITEM  A  IS  FOUND  IN  SYSTEM  B  AND  SYSTEM  C. 

(Cl  TECHNOLOGY  ADVANCES  LEAD  TO  ITEM  A'.  CAN  ALSO  BE  UTILIZED  IN  SYSTEM  D. 
WHICH  REQUIRES  THE  IMPROVED  CHARACTERISTICS.  ITEM  A  IS  DOCUMENTED  BY 
SPEC  00  XXX  A. 

ID)  A  NEW  GENERATION  OF  TECHNOLOGY  LEADS  TO  ITEM  B  WHCH  IS  DEVELOPED  FOR 
SYSTEM  E  AND  DOCUMENTED  BY  SPEC  YYY.  ITEM  B  IS  FUNCTIONALLY  LIKE  ITEM  A' 
BUT  SIGNIFICANTLY  SMALLER  IN  FORM  FACTOR  AND  LESS  EXPENSIVE,  SO  SPEC 
XXXB  IS  DEVELOPED  TO  ADAPT  ITEM  B  TO  SYSTEMS  A,  B,  C,  AND  D  AND  TO  SUPER¬ 
SEDE  SPEC  XXX  AND  SPEC  OOXXX  A.  LATER,  NEW  APPLICATIONS  FOR  ITEM  B  ARE 
FOUND  IN  SYSTEMS  F  AND  G. 


Figure  XXIII-1.  Growth  of  a  standard. 


EFFECTIVENESS 


Parts  management  for  overall  system  effectiveness  is  most  often  cloaked  as 
“good  engineering  practice.”  Tho  performance  of  the  system  hinges  on  proper  parts 
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selection  and  application.  The  reliability  of  the  system  depends  on  (1)  the  inherent 
failure  rate  of  the  selected  parts  and  (2)  the  stresses  put  on  the  parts  by  their  design 
application.  The  availability  of  the  system  depends  on  its  achieved  reliability  and  the 
time  to  restore  it  to  operation  when  a  failure  occurs.  The  availability  of  parts  is  a 
rmyor  factor  in  the  ability  to  repair  a  system;  therefore,  consideration  of  both  current 
and  future  part  availability  is  warranted.  Undoubtedly,  performance  factors  play  a 
rmyor  role  in  part  selection;  a  partial  list  of  these  factors  would  include  size,  weight, 
form  factor,  power  consumption,  environmental  ratings,  and  tolerance.  “Suggestions 
for  Designers  of  Navy  Electronic  Equipment'  (NOSC  TD  250)  provides  many  points 
to  aid  in  avoiding  pitfalls  in  part  selection;  the  appropriate  military  specifications  and 
“selection  and  use'  standards  are  also  useful.  Beyond  these  standard  factors,  parts 
should  be  selected  as  follows: 

1.  Select  high-reliability  parts. 

2.  Use  parts  well  within  their  rated  limits:  derate  the  parts  for  the  environ¬ 
ment  they  must  endure  in  service  (temperature,  EMI,  vibration). 

3.  Choose  parts  whose  dominant  failure  mode  has  minimum  effect  on  the 
equipment. 

4.  Select  parts  and  designs  which  do  not  require  other  components  to  correct 
their  deficiencies  (for  instance,  vibrator-type  power  supplies  will  normally 
require  filtering  to  take  out  the  EMI  they  produce). 

5.  Take  into  account  part  tolerances  and  value  changes  under  stress  and 
aging. 

6.  Choose  parts  which  are  produced  by  mature,  large-volume  manufacturing 
processes  where  other  factors  (size,  weight,  speed,  etc.)  do  not  dictate  a 
less  mature  technology. 

7.  Avoid  sole  source  and  proprietary  parts. 

8.  Conform  to  standardization  steps  4  and  5.  When  specialized  screening  is 
indicated,  cite  the  screening  requirements  which  are  in  excess  of  high  reli¬ 
ability/standard  parts  qualification  requirements. 

EFFICIENCY 

Cost  and  availability  of  parts  are  factors  which  must  always  be  weighed  in 
part  management  decisions.  However,  these  factors  should  not  be  given  primacy  over 
other  part  selection  factors  since  the  initial  cost  of  components  is  only  a  small  portion 
of  an  equipment’s  cost  (typically  10%  for  military  electronics)  and  supposed  savings 
are  quickly  obscured  by  high  support  costs.  Nevertheless,  a  number  of  alternatives 
may  be  available  to  the  project  in  its  parts  decisions. 

1.  In  more  complex  parts  (assemblies  and  units),  off-shelf  items  should  be 
considered  in  preference  to  developing  a  new  item. 

2.  Manufacturers’  high-reliability  lines  of  commercial  parts  are  often  much 
more  readily  available  than  military  parts  and  are  usually  less  expensive; 
these  parts  can  be  used  to  advantage  in  design  and  even  in  limited  produc¬ 
tion  as  long  as  the  military  standard  part  is  used  as  the  provisioning  part 
(see  standardization  step  5). 
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3.  High  reliabilities  meeting  or  exceeding  military  standards  are  often  attain¬ 
able  by  apply!,  g  an  appropriate  screen  to  commercial  parts;  this  applies  to 
piece  parts  and  whole  equipments  alike.  In  applying  screens  to  piece  parts, 
large  enough  volumes  of  parts  must  be  screened  to  amortize  the  screening 
setup  costs  and  the  costs  of  rejected  parts.  When  a  high-reliability/stan¬ 
dard  part  exists,  it  should  be  specified  in  provisioning  documentation. 

4.  When  parts-peculiar  are  justified  (such  as  custom  LSI),  consideration 
should  be  given  to  total  life-cycle  procurement  techniques  to  preclude 
uneconomical  small-lot  reprocurements.  (A  total  life-cycle  procurement 
combines  initial  requirements  and  all  projected  support  requirements.) 

CONFIGURATION  MANAGEMENT 


The  purpose  of  configuration  management  (CM)  is  to  ensure  that  the  knowl¬ 
edge  required  to  design,  operate,  reproduce,  improve,  and  support  a  system  is 
retained.  This  knowledge  is  initially  in  the  minds  of  the  designers,  but  eventually  it 
must  be  documented  and  controlled.  During  design,  task  leaders  should  be  held 
responsible  for  ensuring  that  the  proper  documentation  is  created  and  maintained 
accurately;  this  is  a  function  of  data  management  (see  chapter  XX  and  NAV- 
MATINST  4000.15  (series)).  Once  the  documentation  has  been  produced,  verified, 
and  accepted,  it  becomes  subject  to  the  more  formal  controls  of  configuration  manage¬ 
ment  to  ensure  that  the  equipment  and  its  descriptive  data  are  indeed  of  the  same 
design. 

Configuration  management  should  be  executed  in  accordance  with  SEC- 
NAVINST  4130.2  (series).  CM  is  implemented  through  three  information  baselines: 

Functional  baseline— based  on  the  system  specification 

Allocated  baseline— based  on  detailed  system  partitioning  and  the  specifica¬ 
tions  describing  each  configuration  item 

Product  baseline— based  on  the  established  system  design  implemented  in 
product  design 

The  functional  and  allocated  baselines  consist  of  total  functional  descriptions  of  the 
identified  configuration  items.  The  product  baseline  consists  of  the  total  design  de¬ 
scription  of  all  system  items  down  to,  and  including,  the  level  of  standardization  (see 
chapter  VI);  this  will  normally  consist  of  n  mixture  of  functional  and  fabrication 
specifications,  drawings,  and  associated  lists.  Once  a  baseline  is  established,  it  can 
only  be  changed  through  an  engineering  change  proposal  (ECP).  Two  classes  of  ECP 
and  defined  by  MIL-STD-480— Class  I  and  Class  II.  Class  f  changes  must  be  justified 
on  the  basis  of  their  beneficial  effect  to  the  configuration  item  and  must  be  approved 
by  the  Government  Change  Control  Board  (CCB)  prior  to  implementation  (this  CCB 
will  reside  internally  to  the  project  until  the  production/deployment  phase,  then  the 
CCB  of  the  cognizant  systems  command  will  take  over).  Class  II  changes  are  justified 
only  on  the  basis  of  their  benefits  to  the  originator  qz  the  government  and  may  be 
implemented  as  long  as  they  will  not  be  detrimental  to  the  government  and  when  the 
government  concurs  that  the  ECP  meets  this  criterion.  In  the  implementation  of  CM, 
the  following  recommendations  aro  made: 
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1.  Implement  CM  to  the  level  of  standardization;*  do  not  require  CM  con¬ 
trols  below  that  level. 

2.  Initially  implement  CM  controls  to  the  level  of  repair  until  field  reliability 
objectives  have  been  met;  then  extend  the  level  of  control  as  necessary. 

3.  Defer  invocation  of  CM  controls  on  items  under  a  long-term  warranty  until 
the  warranty  period  is  about  to  end  and  the  government  is  about  to  take 
over  maintenance  from  the  warrantor. 

4.  Defer  full  spares  stocking  until  after  the  product  baseline  is  established  at 
the  level  of  standardization. 

5.  Determine  the  nondetrimental  condition  of  Class  II  changes  on  the  basis  of 
both  performance  and  total  life  cost. 

Additional  CM  policy  for  tactical  system  software  and  ADP  hardware  and  soft¬ 
ware  subject  to  joint  CM  is  covered  in  MIL-STD-2167A.  Project  managers  of  systems 
incorporating  software  should  be  aware  of  the  provisions  of  this  standard. 

Other  references  include: 

MH/-STD-481B 

MIL-STCM82A 

MIL-STD-483A 

MHrSTD-1456 

MIL-STD-1521B 

QUALITY  MANAGEMENT 


Quality  is  defined  as  the  composite  of  all  the  attributes  or  characteristics, 
including  performance  of  an  item  or  product  (source:  DoD  Directive  4155.11).  There 
are  two  mqjor  facets  of  quality  management— quality  assurance  (QA)  and  quality  con¬ 
trol  (QC).  QA  is  a  planned  and  systematic  pattern  of  all  actions  necessary  to  provide 
adequate  confidence  that  an  item  or  product  conforms  to  established  technical 
requirements;  QC  is  the  collection  of  actions  taken  for  the  purpose  of  preventing  pro¬ 
duction  of  defective  materials  or  items  (source:  MIL-STD-109).  Once  technical 
requirements  have  been  derived  from  operational  requirements,  QA  becomes  an 
unavoidable  management  function;  however,  the  imposition  of  formal  QA  procedures 
will  depend  upon  individual  project  requirements. 


*The  level  of  standardization,  as  defined  in  chapter  VI,  will  include  the  lower  level  of 
complexity  of  f  1)  the  level  of  (organic  maintenance)  repair,  or  (2)  the  level  just  below 
the  level  of  design  ownership;  this  will  incorporate  requirements  for  intermediate  and 
depot  level  repair  under  government  cognizance. 
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Categorize  each  technical  requirement  as  either  an  attribute  (it  exists  or  it 
does  not  exist)  or  a  characteristic  (it  exists  in  various  degrees);  further  identify 
v.-hether  the  requirement  is  critical  (unacceptability  is  likely  to,  result  in  a  hazardous 
or  unsafe  condition  for  individuals  using,  maintaining,  or  depending  on  the  product 
or  is  likely  to  prevent  performance  of  the  tactical  mission),  mqjor  (unacceptability  is 
likely  to  result  in  failure  of  the  product  for  its  intended  purpose),  or  minor  (unac¬ 
ceptability  is  unlikely  to  affect  the  usability  of  the  product  for  its  intended  purpose). 
Minimum  acceptable  standards  should  be  established  for  each  characteristic  as  well 
as  desired  levels  of  performance.  The  Section  4  provisions  of  the  system  specification 
should  fully  detail  the  methods  of  inspection  and  test  which  will  be  implemented  to 
assure  the  adequacy  of  the  final  product. 

To  simplify  the  quality  management  problem,  decide  what  attributes  or  char¬ 
acteristics  are  products  of  pure  design  effort  or  subject  to  the  vagaries  of  the  produc¬ 
tion  process.  Of  those  which  are  affected  by  production  factors,  separate  out  the  criti¬ 
cal  or  highly  variable  attributes  and  characteristics.  Design  features  and  many  minor 
requirements  can  be  satisfactorily  inspected  on  a  one-time  basis.  Critical  and  highly 
variable  major  features  should  be  subjected  to  100%  inspection;  other  production- 
affected  features  can  be  subjected  to  sampling  inspections  (see  MIL-STD-105  (attrib¬ 
utes)  and  MIL-STD-414  (variables)).  It  should  be  remembered  that  there  are  many 
components  to  quality— performance,  reliability,  and  workmanship,  for  instance.  An 
item  can  have  high  quality  overall  and  still  be  deficient  in  any  one  area  (like  reliabil¬ 
ity).  It  is  important  to  structure  the  accept/reject  criteria  in  a  manner  such  that  an 
unacceptable  determination  in  any  critical  or  major  feature  results  in  a  rejection.  Do 
not  inspect  or  test  for  features  which  are  not  requirements;  inspection  for  cosmetic 
purposes  only  drives  up  costs. 

The  following  references  may  be  useful  in  imposing  QA/QC  requirements  on 
contractors: 


lock 


MIL-Q-9858A 

MIL-I-45208A 

MIL-STD-45662A 

MIL-STD-2168 

MIL-STD-252B 

MIL-STD-1535A 


Quality  Program  Requirements 
Inspection  System  Requirements 
Calibration  System  Requirements 
Software  QA  Program  Requirements 
Workmanship  Standards 
Supplier  QA  Program  Requirements 


Four  rules  of  analyzing  quality  cost  data: 


1.  If  number  of  dollars  spent  on  corrective  action,  quantity  audits,  etc.,  shows 
decrease  in  scrap-rework  and  other  deviation  costs  >  cost  of  quality  activi¬ 
ties:  continue  spending  otherwise  don’t. 

2.  Publicize  the  results  of  quality  cost  data  throughout  the  project. 

3.  Be  as  careful  of  low  expenditures  for  quality  as  high. 

4.  Spend  money  where  it  counts.  In  general,  if  product  is  80%  of  budget, 
spend  most  quality  money  there. 
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Checklist  for  Quality  Cost  Accounts 
Quality  Creation 

1.  Vendor  control 

2.  Planning-  and  implementing  test;  inspect  and  process  control  procedure 

3.  Design  review 

4.  Training  and  education 

5.  Review  of  material  handling  and  packaging 
Quality  Maintenance 

1.  Calibration  and  repair  of  TE 

2.  Field  testing  of  products 

3.  Procedures  audit 

4.  Failure  analysis 

5.  Data  maintenance 

6.  Audit  of  corrective  action 
Internal  Quality 

1.  Scrap 

2.  Rework 

3.  100%  sorting  of  suspected  material 

4.  Material  review  board  activities 

5.  Production  facilities  downtime  due  to  re-setups 

6.  Extra  vendor  advice 
External  Quality 

X.  Field  complaints 

2.  Customer  allowance  for  rejections 

3.  Product  service  and  repairs 
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DISTRIBUTION  REQUEST 


Please  send  copy  (copies)  of  PROJECT  MANAGEMENT  AND  SYSTEMS  ENGI¬ 
NEERING  GUIDE  (NOSC  TD  108)  to: 

(Print  or  Type) 

Name 

Title 

Company 

Address 

City  State  Zip 

Comments: 
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Place 

Stamp 

Here 


COMMANDER 

NAVAL  OCEAN  SYSTEMS  CENTER 

271  Catalina  Boulevard 

SAN  DIEGO,  CALIFORNIA  92152 

ATTN:  TECHNICAL  PUBLICATIONS  DISTRIBUTION 
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APPENDIX  A:  TELCAM  ENVIRONMENTAL  STUDY 


Under  the  present  system  ofhardware  acquisition,  all  equipments  which  are 
designated  for  Navy  shipboard  use  are  supposed  to  be  subjected  to  the  analyses  and 
testing  rigors  specified  in  MIL-E-16400.  A  review  of  the  requirements  of  MIL¬ 
E-16400  in  relation  to  measured  shipboard  data  indicates  that  savings  in  time  and 
money  can  be  gained  by  reducing  environmental  analyses  and  requirements  testing. 
Before  proceeding  further,  it  should  be  stated  and  emphasized  that  this  study  does 
not  apply  to  critical  equipment  (critical  equipments  being  those  without  which  the 
ship  cannot  operate  or  perform  its  mission).  Any  reduction  of  environmental  require¬ 
ments  should  be  based  upon  the  elimination  or  modification  of  those  tests  and  analy¬ 
sis  requirements  pertaining  to  shipboard  conditions  deemed  to  be  abnormal;  that  is, 
conditions  occurring  very  rarely  or  never  having  occurred  to  date  to  an  operational 
Navy  ship.  Examples  of  such  conditions  would  be  a  nearby  underwater  blast,  nuclear 
air  blast,  and  simultaneous  high  temperature  and  high  relative  humidity.  The  sec¬ 
tions  of  MIL-E-16400  describing  environmental  tests  and  analyses  were  reviewed  to 
determine  (1)  which  would  be  classed  as  abnormal  occurrences  not  applicable  to  non- 
critical  equipments  and  which  could  be  modified  to  more  closely  represent  the 
expected  or  known  shipboard  conditions,  and  (2)  which  appear  adequate  in  their  pre¬ 
sent  form.  Those  which  appear  adequate  and  those  which  could  be  modified  would 
necessarily  be  applicable  to  all  equipments  to  assure  satisfactory  operation  aboard 
ship.  The  environmental  tests  and  analyses  describing  shipboard  conditions  occurring 
very  rarely  or  never  having  occurred  to  date  to  an  operational  Navy  ship  are  nuclear 
air  blast  analysis  and  shock  tests  so  designed  as  to  simulate  shock  created  by  a 
nearby  underwater  blast.  This  shock  test  is  specified  in  MIL-S-901.  Both  could  be 
eliminated  from  MIL-E-16400  as  requirements  for  noncritical  equipments.  It  has  also 
been  noted  that  acceptable  performance  under  vibration  testing  has  been  indicative  of 
satisfactory  shock  performance. 

One  type  of  shock  occurring  aboard  ships  that  is  not  addressed  in  MIL-E-16400 
is  shock  created  by  the  muzzle  blast  from  a  ship’s  own  guns  impinging  on  external 
bulkheads.  The  severity  of  this  type  shock  is  dependent  upon  the  distance  from  the 
gun  muzzle  to  a  bulkhead  and  the  distance  from  this  bulkhead  at  which  equipment  is 
mounted.  In  the  past,  attempts  have  been  mode  to  analytically  qualify  equipment  to 
this  type  shock  with  little  or  no  success.  At  times  individual  equipment  specifications 
will  require  installation  aboard  ships  in  a  specified  area  with  equipment  performance 
being  monitored  during  gunfire  exercises. 

Certain  tests  and  analyses  describe  conditions  which  can  be  expected  to  occur 
frequently  aboard  Navy  ships  and  should  be  applied  to  all  equipments.  These  are 
wind  speed  analysis  and  icing,  noise,  and  enclosure  tests.  Airborne  and  struc- 
tureborne  noise  and  enclosure  tests  should  be  applied  to  all  sheltered  equipments. 
These  tests  and  analyses  are  appropriate  in  their  present  form  and  should  be  applied 
to  all  equipments  as  specified. 

MIL-E-16400  requires  all  equipments  to  be  inclination  tested.  This  test  should 
be  required  only  for  equipments  deemed  to  be  position  sensitive  (equipments  whose 
proper  operation  is  dependent  upon  fluid  levels,  position-sensitive  switches,  etc.), 
since  this  test  is  a  slow  rocking  motion  to  which  most  equipments  are  insensitive. 
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Vibration,  temperature,  and  relative  humidity  are  conditions  which  exist 
aboard  ship.  MIL-E-16400  requires  all  equipments  to  be  vibration  tested  as  specified 
in  MIL-STD-167.  Temperature  and  relative  humidity  tests  are  specified  in  MIL¬ 
E-16400.  Since  these  conditions  constantly  exist,  it  is  mandatoiy  that  all  equipments, 
both  critical  and  noncritical,  be  capable  of  satisfactoiy  operation  over  an  extended 
period  of  time  while  being  exposed  to  them.  Therefore,  vibrational  envelopes  were 
established  for  various  classes  of  Navy  ships,  and  worldwide  extreme  temperature  and 
relative  humidity  conditions  were  identified. 

As  a  first  step  toward  establishing  environmental  vibration  envelopes  for  vari¬ 
ous  classes  of  Navy  ships,  literature  on  hand  was  reviewed.  This  encompassed  48 
annual  vibration  survey  reports  from  the  Boston,  Long  Beach,  Norfolk,  Pearl  Harbor, 
Philadelphia,  Portsmouth,  Puget  Sound,  San  Francisco  Bay,  and  Charleston  Naval 
Shipyards  which  covered  the  period  from  1961  to  1973.  From  these  reports  vibration 
data  from  more  than  160  ships  of  the  DD,  DDG,  DLG,  DL,  DE,  DEC,  CVS,  CVA, 
CVAN,  LPH,  LST,  MSO,  PG,  and  SSN  types  were  selected.  NOSC  reports  1577  and 
1701  which  are  accumulations  of  vibration  data  measured  by  NELC  personnel  aboard 
various  Navy  ships,  and  Naval  Ship  Research  and  Development  Center  (NSRDC) 
Reports  2338  and  2338A  were  also  reviewed.  Only  vibration  data  for  spaces  contain¬ 
ing  or  in  close  proximity  to  electronic  equipment  were  selected,  and  these  electronic 
equipment  spaces  were  in  the  ship’s  superstructure.  Since  literature  surveys  show  the 
most  severe  vibration  conditions  occur  in  a  ship’s  stern,  the  vibration  data  selected 
do  not  represent  the  worst-case  conditions  occurring  aboard  ship.  This  survey  also 
indicates  that  the  shock  and  vibration  environment  existing  on  a  ship’s  mast  is  mini¬ 
mal,  and  these  data  were  not  used. 

The  vibration  data  were  sorted  by  ship  typo  and  converted  to  displacement 
(inches  single  amplitude)  vs  frequency  (Hz).  A  plot  of  these  data  was  made  for  each 
class  of  ship  (fig  A-l  through  A-8)  by  plotting  data  points  on  multicoordinate  vibra¬ 
tion  graph  paper  and  connecting  peak  values  with  straight  lines.  The  region  under 
the  plotted  curve  was  then  considered  to  be  the  vibration  envelope,  that  is,  any  point 
within  the  curve  may  be  a  vibration  condition  expected  aboard  that  type  of  ship.  Re¬ 
ferring  to  figures  A-l  through  A-8,  the  frequencies  range  from  1  to  45  Hz  with  accel¬ 
erations  seldom  exceeding  0  2  g  and  displacements  seldom  exceeding  0.030  inch  single 
amplitude. 

See  figures  A-9  and  A- 10  for  TELCAM  vibration  test  amplitudes,  figure  A-ll  for 
a  MIL-STD-167  vibration  test  amplitude.  The  TELCAM  vibration  test  amplitudes  are 
listed  in  table  A-l. 
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Flgtiro  A-2.  DOG-OD  TELCAM  vibration  limits. 


Figure  A-5.  LPH  TELCAM  vibration  limits. 
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Figure  A-8.  PG  TEICAM  vibration  limits. 


Table  A-l.  Input  amplitudes. 


Use  A  or  B,  whichever  is  better  suited  to  the  vibration  test  machine  in  use. 
A  (Constant  Displacement)  (see  fig'.  A-9) 


Freotiencv  Ranee  (Hz) 

Single  Amplitude  finch! 

g=  A/G 

4-10 

.03±.003 

.05-.3 

10-17 

.Oldt.001 

.10-.3 

17-30 

.003±.0003 

.09-.13 

30-60 

,001±.0001 

.09-.38 

B  (Constant  Acceleration)  (see  fig.  A-10) 

Frequency  Range  (Hz) 

Single  Amplitude  finch) 

g  =  A/G 

4-8 

.03±.003 

.05-.2 

8-60 

.03-.0005 

,2±.02 

The  following  vibration  tests  were  developed  from  the  vibration  envelope  infor¬ 
mation: 

SCOPE.  All  communications  equipment  intended  for  shipboard  use  under  the  TEL- 
CAM  concept  shall  be  proved  acceptable  under  the  vibration  tests 
described  herein.  Magnitudes  of  input  motions  and  the  associated  frequencies  are 
chosen  to  represent  usual  conditions  existing  aboard  ship  and  do  not  cover  the  vibra¬ 
tion  environment  that  may  exist  when  a  ship  has  sustained  battle  damage  and  must 
continue  to  operate.  Users  of  this  test  are  therefore  encouraged  to  be  critical  in  judg¬ 
ing  acceptability  of  the  equipment  under  test. 

BASIS  OF  ACCEPTABILITY.  Acceptability  shall  be  judged  as  under  MIL-STD-167B 
(SHIPS),  Mechanical  Vibrations  of  Shipboard  Equipment: 

Basis  of  acceptability.  Acceptability  will  be  contingent  upon  the 
ability  of  the  equipment  to  perform  its  function  during  and  after 
the  tests  specified  in  5.1.3.  Minor  damage  or  distortion  will  be 
permitted  during  test  providing  such  damage  or  distortion  does 
not  in  any  way  impair  the  ability  of  the  equipment  to  perform 
its  principal  functions.  Because  of  the  numerous  types  of  equip¬ 
ment  covered  by  this  standard,  a  definite  demarcation  between 
mqjor  and  minor  failures  cannot  be  specified.  Therefore,  such 
decisions  must  necessarily  be  left  to  the  judgment  of  the  test 
engineer.  In  general,  a  mqjor  failure  is  one  which  would  cause 
malfunction  of  the  item  of  equipment  for  a  long  period.  Non- 
repetitive  failures  of  such  parts  as  vacuum  tubes,  condensers 
and  wiring,  which  can  be  easily  replaced  or  repaired,  are  gener¬ 
ally  considered  minor  failures.  As  such,  the  repair  could  be 
made  and  the  test  continued  with  no  penalty  to  the  remainder 
of  the  equipment.  Sometimes  the  critical  use  of  the  equipment 
will  determine  the  category  of  failure;  that  is,  a  failure  of  a  part 
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ia  a  lighting  circuit  may  be  considered  minor.  The  same  failure 
in  a  reactor  control  circuit  may  be  mqjcr.  Thus  the  test  engi¬ 
neer,  or  command  or  agency  concerned,  shall  be  responsible  for 
specifying  a  major  or  minor  failure. 

In  addition,  large  amplifications  of  motion  created  by  resonant  response  of  the  equip¬ 
ment  under  test  are  to  be  considered  unacceptable  unless,  in  the  opinion  of  the  vibra¬ 
tion  test  engineer,  such  responses  do  not  predicate  early  equipment  failure. 

INTENT  OF  TESTS.  Vibration  tests  specified  herein  are  intended  to  locate  reso¬ 
nances  of  the  equipment  under  test,  and  to  impose  a  2-hour  endurance  test  at  those 
frequencies  and  amplitudes. 

TESTING  MACHINES.  Any  vibration  testing  machine  capable  of  providing 
sinusoidal  motion,  of  the  frequency  and  amplitudes  specified,  to  the  points  of  attach¬ 
ment  of  the  equipment  under  test  may  be  used.  Means  shall  be  provided  for  control¬ 
ling  the  direction  of  vibration  of  the  testing  machine  and  for  adjusting  and  measuring 
its  frequencies  and  amplitude  of  vibration  to  keep  them  within  prescribed  limits.  If 
the  lower  frequency  limit  of  4  Hz  specified  cannot  be  reached,  the  available  machine 
may  be  used  upon  approval  of  the  command  or  agency  concerned  provided  the  natural 
frequencies  of  the  equipment  in  translational  and  rocking  modes  of  vibration  do  not 
lie  below  the  lowest  frequency  of  the  available  testing  machine.  This  may  sometimes 
be  determined  by  properly  controlled  transient  excitation,  such  as  bumping  the  equip¬ 
ment  to  see  whether  low-frequency  resonances  exist.  In  no  case  shall  a  vibration  test¬ 
ing  machine  be  used  which  has  a  minimum  frequency  greater  than  10  Hz. 

METHODS  OF  ATTACHMENT.  For  all  vibration  tests,  the  equipment  shall  be 
secured  to  the  vibration  machine  in  the  same  manner  in  which  it  will  be  secured  on 
shipboard.  If  alternative  mountings  of  the  equipment  are  possible  and  contemplated, 
a  complete  TELCAM  test  shall  be  conducted  in  each  alternative  mounting  configura¬ 
tion. 

FIXTURES.  When  fixtures  must  be  used  in  attaching  equipment  to  the  vibration 
machine  (for  instance,  when  the  equipment  requires  bulkhead  or  overhead  support), 
the  fixtures  used  shall  be  sufficiency  rigid  to  ensure  that  motion  at  the  points  of 
attachment  of  the  equipment  is  essentially  the  same  as  motion  of  the  main  machine 
platform. 

PORTABLE  EQUIPMENT,  Equipment  which  is  not  intended  for  permanent  attach¬ 
ment  on  shipboard  shall  be  attached  to  the  vibration  testing  machine  >n  a  manner 
representative  of  that  in  which  it  will  be  stored  on  board  ship.  In  many  cases,  this 
will  require  that  the  equipment  be  restrained  on  the  vibration  testing  machine  by 
means  of  suitable  small  woven  straps  or  line. 

ORIENTATION  FOR  VIBRATION  TEST.  Equipment  shall  be  installed  on  vibration 
testing  machines  in  such  manner  that  the  direction  of  vibration  will  be  in  turn  along 
each  of  the  three  rectilinear  orientation  axes  of  the  equipment  as  installed  on  ship¬ 
board  —  vertical,  athwartship,  and  fore  and  aft.  On  a  horizontal  vibration  testing 
machine,  the  equipment  may  be  turned  90  degrees  in  the  horizontal  plane  in  order  to 
vibrate  it  in  each  of  the  two  horizontal  orientations.  At  no  time  shall  the  equipment 
be  installed  in  any  other  way  than  its  normal  position.  Because  no  particular  orienta 
tion  of  portable  equipment  can  be  considered  normal,  orientation  of  portable  equip¬ 
ment  for  vibration  testing  shall  bo  left  to  the  discretion  of  test  engineer. 


RESILIENT  MOUNTINGS.  Equipment  which  is  to  be  installed  on  resilient  mounts 
on  board  ship  shall  be  installed  on  those  mounts  for  vibration  testing.  Resilient 
mountings  integral  to  the  equipment  (intended  to  protect  sensitive  devices  within  the 
equipment)  shall  be  left  in  place. 

VIBRATION  TESTS.  Each  of  the  tests  specified  herein  shall  be  conducted  separately 
in  each  of  three  principal  axes  of  the  equipment  under  test.  Each  type  of  test 
(exploratory  vibration,  variable  frequency,  and  endurance,  below)  shall  be  completed 
in  all  three  axes  before  proceeding  to  the  next  type  of  vibration  test.  During  the  tests, 
the  equipment  shall  be  secured  to  the  test  machine  and  oriented  as  specified  in 
METHODS  OP  ATTACHMENT  and  FIXTURES,  above.  The  equipment  shall  be 
energized  and  performing  its  normal  functions.  Appropriate  test  equipment  shall  be 
applied  to  the  equipment  under  test  to  reveal  vibration-caused  electrical  malfunctions 
which  are  not  manifested  in  mechanical  responses.  Unless  otherwise  directed,  the  test 
shall  be  discontinued  if  major  failure  occurs,  and  the  entire  vibration  test  series 
repeated  following  repair  or  correction  of  deficiencies.  An  entirely  new  test  specimen 
may  be  substituted  for  the  retest,  hut  the  substitution  must  be  noted  in  the  test 
report  furnished.  (See  TEST  REPORT,  below.) 

EXPLORATORY  VIBRATION  TEST.  To  determine  the  presence  of  possibly  harmful 
resonances  in  the  equipment  under  test,  vibration  in  the  frequency  range  from  4  Hz 
to  the  lowest  attainable  frequency  not  to  exceed  10  Hz)  to  60  Hz  shall  be  applied. 
Amplitude  of  input  vibration  shall  be  only  great  enough  to  determine  resonant 
responses  of  the  equipment  under  test  and  to  obtain  an  approximation  of  the  amplifi¬ 
cation  factor.  In  no  case  shall  input  amplitudes  exceed  those  specified  for  the  VARI¬ 
ABLE  FREQUENCY  TEST,  below.  The  frequency  range  is  to  be  swept  from  low  to 
high  frequency  at  a  rate  not  greater  than  4  Hz  per  minute  nor  less  than  1  Hz  per 
minute.  Resonant  frequencies  and  approximate  amplification  factors  shall  be  noted.  If 
believed  necessary  by  the  test  engineer,  resonances  noted  during  the  scan  shall  be 
reestablished,  peaked,  and  held  briefly  to  permit  adequate  documentation.  Reso¬ 
nances,  how  they  were  manifest,  their  frequencies  (±.0.5  Hz),  and  their  approximate 
amplification  factors  shall  be  included  in  the  test  report.  (See  TEST  REPORT, 
below.) 

VARIABLE  FREQUENCY  TEST.  The  equipment  under  test  shall  be  subjected  to 
vibration  between  4  Hz  (or  the  lowest  attainable  frequency  not  to  exceed  10  Hz)  and 
60  Hz  with  input  amplitudes  as  shown  in  table  A-l.  Vibration  input  shall  be  at  dis¬ 
crete  integral  frequencies  (±.0.5  Hz),  and  each  frequency  shall  be  maintained  for  a 
period  of  5  minutes.  Peak  responses  of  the  equipment  under  test  are  likely  to  occur  at 
nonintegral  frequencies.  Mechanical  nonlinearity  of  the  equipment  under  test  will 
cause  a  shift  in  the  frequency  of  maximum  response  as  documented  in  EXPLORA¬ 
TORY  VIBRATION  TEST,  above.  Therefore,  as  frequency  changes  are  made,  the  test 
engineer  shall  carefully  observe  the  equipment  under  test  and  peak  any  response, 
even  though  it  may  occur  at  a  nonintcgral  frequency,  hold  the  response  long  enough 
to  obtain  documentation,  and  then  move  on  to  the  next  integral  frequency. 

ENDURANCE  TEST.  The  equipment  under  test  shall  be  vibrated  for  a  period  of  2 
hours  at  each  of  the  resonances  noted  in  VARIABLE  FREQUENCY  TEST,  with  input 
amplitudes  as  specified  in  table  A-l.  In  the  event  no  resonances  have  been  noted,  the 
equipment  shall  be  vibrated  for  2  hours  at  60  Hz. 
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TEST  REPORT.  The  test  report  to  be  furnished  the  command  or  agency  concerned  by 
the  testing  laboratory  shall  include  detailed  descriptions  of  any  damage  or  malfunc¬ 
tioning  incurred  and  at  what  stage  in  the  tests  it  occurred.  When  possible,  photo¬ 
graphs  of  physical  damage  shall  be  included.  Recommendations  are  desired  as  to  what 
corrective  measure,  if  any,  should  be  taken.  It  shall  also  include  other  pertinent  infor¬ 
mation,  such  as  the  overall  dimensions  of  the  equipment,  its  weight,  approximate  lo¬ 
cation  of  the  center  of  gravity,  a  sketch  or  photograph  of  the  methods  used  in 
mounting  it  on  the  test 

machines,  a  listing  of  resonances,  frequencies  at  which  resonances  occurred,  and  their 
approximate  amplification  factors. 

A  temperature/relative  humidity  profile  was  developed  for  TELCAM  much  like 
the  TELCAM  vibration  profile.  Data  that  had  previously  been  collected  from  instru¬ 
mented  shipboard  surveys  and  weather  reports  had  been  plotted  as  temperature  vs 
relative  humidity  and  were  compared  with  MIL-E-16400  requirements  as  shown  in 
figure  A- 12.  The  bounds  of  the  natural  occurrences  plotted  became  the  TELCAM  lim¬ 
its.  Referring  to  figure  A-13,  an  easily  applied  test  can  be  described  by  a  temperature/ 
relative  humidity  trapezoid  with  corner  points  at  68  degrees  F  with  95-percent 
relative  humidity,  95  degrees  F  with  95-percent  relative  humidity,  122  degrees  F  with 
65-percent  relative  humidity,  and  68  degrees  F  with  65-percent  relative  humidity. 
This  test  is  conducted  by  making  five  complete  cycles  around  the  trapezoid  starting 
and  finishing  at  95  degrees  F  and  95-percent  relative  humidity.  Each  test  condition  is 
maintained  for  5  hours  with  1  hour  allowed  for  the  transition  between  points, 
thereby  making  each  cycle  24  hours  long.  This  test  differs  from  MIL-E-16400 
requirements  in  that  it  does  not  require  the  simultaneous  high-temperature/high- 
relative  humidity  condition  of  122  degrees  F  with  95-percent  relative  humidity.  This 
condition  does  not  exist  naturally.  The  lower  temperatures  and  transitions  simulate 
conditions  existing  aboard  ship  when  air  conditioning  equipment  fails  and  a  commu¬ 
nications  compartment  must  be  opened  to  ambient  temperature  and  relative  humidity 
levels. 

Both  the  TELCAM-developed  vibration  and  temperature/relative  humidity  tests 
are  not  as  severe  as  their  parent  MIL  tests.  This  condition  resulted  because  the  TEL¬ 
CAM  tests  were  not  developed  to  test  equipment  for  survivability  under  abnormal 
shipboard  environments,  but  to  provide  confidence  the  equipment  will  endure  the 
usual  environments.  All  shipboard  mounted  equipments  are  constantly  exposed  to  the 
vibration  and  temperature/relative  humidity  environments  during  at-sea  operation, 
therefore,  testing  to  and  determining  equipment  survivability  under  these  conditions 
should  be  considered  as  minimum  requirements.  Equipment  should  be  tested  to  the 
inclination,  electrical  transient,  noise,  and  enclosure  tests,  as  applicable,  specified  in 
MIL-E-16400  to  increase  the  confidence  level  of  shipboard  survivability. 

To  date,  four  commercial  equipments  have  been  chosen  for  TELCAM  vibration 
and/or  temperature/relative  humidity  testing,  These  were  a  floppy  disk  memory  unit, 
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Figure  A-12.  Actual  MIL-E-16400  tomperalure/relative  humidity  test. 


Figure  A-13.  TELCAM  temperatui e/relative  humidity  test  (revised). 


a  digital  plotter,  a  cassette  tape  recorder,  and  a  single  sideband  radiotelephone.  The 
radiotelephone  has  not  been  completely  tested  due  to  initial  problems  requiring  cor¬ 
rection  by  the  manufacturer.  All  the  equipments  selected  for  testing  under  the  TEL- 
CAM  program  are  assumed  to  be  readily  available  so  that  redundant  equipment  can 
be  on  hand.  All  tests  that  will  be  applied  to  these  equipments  represent  noncombat¬ 
ant  situations. 

The  floppy  disk  memoiy  unit  was  selected  for  testing  because  it  is  an  ideal  can¬ 
didate  for  use  with  shipboard  minicomputers.  To  monitor  test  performance,  alpha¬ 
betical  characters  from  A  through  P  were  stored  on  the  inside  track  of  the  disk.  The 
characters  were  transferred  from  the  inside  track  to  the  outside  track,  printed  out  via 
teletype,  and  then  compared  with  the  inside  track.  This  was  done  continuously  during 
vibration  testing  and  at  the  end  of  each  temperature/relative  humidity  24-hour  cycle. 
When  a  discrepancy  occurred,  the  teletype  printed  out  DISK  NOT  READY. 

The  floppy  disk  memoiy  unit  was  first  subjected  to  the  TELCAM  vibration  test. 
The  constant  displacement  method  was  used.  The  only  vibration-related  discrepancy 
noted  during  the  test  was  a  resonance  in  the  circuit  board  occurring  at  51  Hz.  This 
condition  was  eliminated  by  attaching  the  circuit  board  corners  to  the  chassis  Two 
DISK  NOT  READY  messages  were  printed  out  by  the  teletype  during  the  test,  but 
they  were  not  repeatable,  and  no  vibration-related  reason  could  be  found. 

The  TELCAM  temperature/relative  humidity  test  was  conducted.  The  test  was 
started  at  95  degrees  F  and  65-percent  relative  humidity.  As  stated,  the  alphabetical 
characters  were  transferred  from  the  inside  track  to  the  outside  track,  printed  out  by 
the  teletype,  and  compared  with  the  inside  track  at  the  end  of  each  24-hour  cycle.  The 
unit  was  turned  off  during  the  24-hour  cycle.  When  the  unit  was  turned  on,  the  disk 
required  approximately  30  seconds  to  warm  up  before  it  would  respond  to  the  teletype 
control.  Moisture  collecting  inside  the  unit  that  had  to  be  exhausted  by  the  unit’s  in¬ 
ternal  cooling  fan  before  proper  operation  could  be  obtained  was  considered  the  cause 
for  this  30-socond  delay. 

To  increase  the  confidence  level  that  the  floppy  disk  memoiy  unit  would  survive 
aboard  ship,  the  supply  line  voltage  and  frequency  tests  described  in  MIL-E-16400F 
section  4.5.4  were  performed.  No  abnormal  effects  were  apparent  during  the  steady- 
state  and  transient  tests.  The  teletype  did  print  out  the  DISK  NOT  READY  message 
during  power  dropout  tests.  This  condition  was  deemed  insignificjnt  as  it  occurred 
only  when  the  power  dropped  out  exactly  as  its  waveform  crossed  the  neutral  axis, 
and  this  would  rarely  occur  naturally.  This  condition  had  no  detrimental  effect  on  the 
unit,  which  operated  properly  when  the  power  returned  to  normal 

These  tests  performed  on  the  floppy  disk  memoiy  unit  indicate  that  it  should 
deliver  satisfactory  performance  if  installed  aboard  Navy  ships. 

The  digital  plotter  was  submitted  for  vibration  testing  by  the  Naval  Undersea 
Center  (NUC).  The  plan  was  to  perform  an  exploratory  vibration  test  (per  MIL- 
STD-167)  only  in  each  of  the  three  mutually  perpendicular  axes  to  identify  resonant 
frequencies. 

During  actual  testing,  input  levels  had  to  be  reduced  when  resonances  were 
reached  to  avoid  damaging  the  plotter.  These  reduced  levels  were  compared  with  and 
were  equal  to  or  slightly  larger  than  the  TELCAM  vibration  levels.  Throughout  this 
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exploratory  vibration  test  the  plotter  operated  properly,  and  at  no  time  did  the  trace 
become  discontinuous  or  rippled.  The  tracing  capability  of  the  plotter  was  of  prime 
importance  during  the  test.  The  plotter  can  be  considered  acceptable  insofar  as  vibra¬ 
tion  is  concerned,  but  before  any  conclusions  can  be  reached  as  to  the  overall  suitabil¬ 
ity  of  this  plotter  for  shipboard  use,  further  testing  and  analysis  will  he  required. 

The  cassette  tape  recorder,  which  is  being  used  in  a  shore-based  Navy  receiver, 
was  subjected  to  the  TELCAM  temperature/relative  humidity  and  vibration  tests. 
Performance  was  measured  during  testing  by  making  voice  recordings  and  by  operat¬ 
ing  playback,  fast  forward,  and  fast  reverse  controls. 

The  TELCAM  temperature/relative  humidity  test  was  run  for  four  complete 
cycles  (96  continuous  hours).  Recorder  performance  was  monitored  just  as  the  95 
degree  F  and  65-percent  relative  humidity  condition  was  reached.  The  recorder  was 
turned  off  during  the  rest  of  the  cycle.  The  only  abnormality  noted  was  slight  slug¬ 
gishness  in  the  reverse,  or  rewind,  mode  of  operation.  This  sluggishness  was  not 
deemed  detrimental  to  recorder  performance,  and  it  disappeared  when  the  recorder 
was  returned  to  room  temperature. 

The  constant  displacement  with  constant  level  crossover  TELCAM  vibration 
test  (see  fig.  A- 10)  was  used  to  test  the  recorder  in  the  vertical  axis  During  the  com¬ 
plete  exploratory,  variable  frequency,  and  endurance  tests  no  resonances,  perform¬ 
ance  degradation,  or  other  discrepancies  were  noted. 

Since  the  recorder  performed  satisfactorily  during  TELCAM  vibration  testing 
in  the  vertical  axis,  the  more  severe  requirements  of  MIL-STD-167  were  applied  in 
the  remaining  vibration  test.  Temperature/relative  humidity  and  vibration  test 
results  indicate  this  recorder  should  operate  satisfactorily  aboard  ship,  even  under 
abnormal  vibration  conditions. 

The  literature  survey  of  shipboard  environmental  conditions  is  continuing. 
Other  equipments  are  being  investigated  as  possible  candidates  to  be  exposed  to  TEL- 
CAM-developed  tests,  and,  where  no  environmental  data  exist,  the  possibility  of  per¬ 
forming  at-sea  measurements  is  being  considered. 

New  ship  classes  such  as  the  Newport  class  of  LST,  LCC,  1000  series  DE,  etc., 
should  be  surveyed  and  underway  measurements  made  to  document  their  environ¬ 
ments.  (See  Jane’s  Fighting  Ships.)  Equipment  packaging  and  transportation  envi¬ 
ronments  have  not  been  investigated.  Many  instances  have  occurred  in  which 
equipment  has  been  damaged  during  transportation  due  to  inadequate  packaging  and 
rough  handling.  Time  and  effort  should  be  expended  to  eliminate  some  of  the  inade¬ 
quacies  in  these  areas. 
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APPENDIX  B:  RECOMMENDATIONS  OF  GOVERNMENT 
STUDIES  ON  REDUCING  COSTS  OF  ELECTRONIC 
SYSTEMS  AND  EQUIPMENT 


DISCUSSION 


A  number  of  government  studies  have  sought  answers  to  the  problem  of  reduc¬ 
ing  the  costs  of  acquiring  and  using  military  items.1'*  It  was  originally  intended  here 
to  present  a  matrix  of  recommendations  vs  each  study  to  provide  an  overview  of  ways 
and  means  for  cost  reduction.  However,  the  most  recent  study  to  date  with  a  pub¬ 
lished  report  has  incorporated  the  results  of  the  other  studies  with  those  of  its  own, 
and  thus  represents  current  and  official  thinking  across  the  board  on  cutting  elec¬ 
tronic  system/equipment  costs.  It  is  the  “Electronics-X”  study.  In  addition,  Electron- 
ics-X  is  especially  pertinent  and  significant  to  the  TELCAM  project  since  it  addresses 
itself  specifically  to  electronic  systems  and  equipment,  whereas  the  other  studies 
either  are  of  a  very  general,  statute-oriented  nature  [e.g.,  the  “Report  of  the  Commis¬ 
sion  on  Government  Procurement”)  or  focus  on  some  one  aspect  of  reducing  costs 
(e.g.,  the  design-to-cost  emphasis  of  the  “Report  of  the  Defense  Science  Board  Task 
Force  on  Reducing  Costs  of  Defense  System  Acquisitions").  Therefore,  rather  than  a 
matrix,  a  tabulation  of  Electronics-X  recommendations  is  presented.  In  the  few 
instances  where  other  study  recommendations  pertinent  to  TELCAM  are  not  included 
among  Electronics-X  recommendations,  they  are  additionally  listed  and  specially 
noted.  This  tabulation  may  be  considered  all-inclusive  with  respect  to  current  govern¬ 
ment  study  recommendations  for  reducing  the  costs  of  acquiring  and  using  electronic 
systems  and  equipment. 


'Report  R-195,  Institute  for  Defense  Analyses,  Science  and  Technology  Division,  Elec¬ 
tronics-X:  A  Study  of  Military  Electronics  with  Particular  Reference  to  Cost  and  Reli¬ 
ability,  January  1974,  prepared  for  Defense  Advanced  Research  Projects  Agency, 
DDR&E 

2Report  of  the  Commission  on  Government  Procurement,  December  1972,  to  the  Con¬ 
gress  of  the  United  States 

3 Report  of  the  Defense  Science  Board  Task  Force  on  Weapon-System  Simplification, 
Summer  of  1970,  DDR&E 

4 Report  of  the  Defense  Science  Board  Task  Force  on  Avionics,  February  1973,  DDR&E 
5 Report  of  the  Defense  Science  Board  Task  Force  on  Reducing  Costs  of  Defense  System 
Acquisitions,  March  15, 1973,  DDR&E 

eCost  Growth  in  Mqjor  Weapons  Systems,  March  26, 1973,  report  by  the  Comptroller 
General  of  the  United  States  to  the  Committee  on  Armed  Services,  House  of  Repre¬ 
sentatives 
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RECOMMENDATIONS 


COST-DATA  COLLECTION  AND  REPORTING  SYSTEMS 

1.  *“  A  systematic  effort  should  be  undertaken  to  develop  a  step-wise 
implementation  of  a  complete  and  uniform  cost  accounting  system  through¬ 
out  DoD,  with  emphasis  on  valid  input  data.  As  a  first  step,  a  marginal  cost 

system  using  sampling  techniques  for  support-cost  inputs  should  be  imple¬ 
mented.  The  system  must  then  evolve  to  cover  full  costs  of  both  acquisition 
and  support. 

2.  **A  central  organization  within  OSD  should  be  designated  to  organize  this 
cost  information  system  and  to  coordinate  the  efforts  of  responsible  service 
elements. 

3.  To  test  and  exercise  the  system,  each  service  mqjor  procurement  command 
should  designate  certain  electronic  systems  for  review  of  cost  reporting 
requirements.  Appropriate  steps  should  be  taken  to  ensure  consistency 
among  the  report  outputs,  complete  record  retrieval,  and  periodic  validation 
of  the  reported  costs.  These  records  should  be  centrally  located  and  should 
be  made  available  to  the  cost  analysis  community. 

RELIABILITY  DATA  COLLECTION  AND  REPORTING  SYSTEMS 

4.  ***In  each  major  producer  command  (AMC,  Navy  SYSCOMs,  AFSC, 

AFLC),  establish  (or  broaden)  a  system  for  competent  technical  reporting  of 
reliability,  availability,  and  maintainability  (RAM)  and  marginal  cost  feed¬ 
back  information  from  the  field  on  selected  systems  and  equipments,  using 
sampling  methods. 

5.  ‘‘Organize  a  RAM  Data  Systems  Task  Force,  representing  the  several  ser¬ 
vices  and  chaired  by  OSD,  to  study  and  compare  the  relative  cost- 
effectiveness  of  a  routine  maintenance  data  collection  system  (such  as  3-M 
or  AFM  6G-1)  with  a  sampled-data  collection  system  (such  as  TAMMS-SDC). 

6.  Establish  a  new  RAM  Information  Exchange  Program  at  the  electronic 
equipment  level  in  a  form  patterned  after  the  Government-Industry  Data 
Exchange  Program  (GIDEP). 


“High  priority 
‘“Highest  priority 
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REQUIREMENTS  AND  ACQUISITION  DECISIONS 


*7.  ***In  exploring  and  establishing  a  system  requirement,  give  performance, 

physical  characteristics,  cost,  quantity,  and  schedule  equal  status  from  the 
beginning,  and  perform  tradeoffs  among  these  early  in  the  game. 

8.  **•  In  m^jor  system  developments,  separate  vehicle  and  electronic  subsys¬ 
tem  initial  operational  capabilities,  where  possible,  and  develop  the  elec¬ 
tronics  independently.  Consolidate  like  subsystem  or  equipment  require¬ 
ments  wherever  feasible. 

9.  ***  Increase  visibility  to  top-level  management  of  potentially  cost-driving 
developments  of  electronic  subsystems  associated  with  major  systems,  by 
instituting  suitable  review  prior  to  each  DSARC  review.  As  appropriate, 
provide  for  a  similar  visibility  to  managers  of  developments  of  less-tban- 
mfjor  electronic  subsystems  and  equipments  by  refocusing  reviews  to  make 
them  analogous  to  DSARC  reviews,  but  at  lower  management  levels. 

*10.  ***  Give  increased  consideration  to  product  improvement  programs  as  a 

means  of  fulfilling  new  requirements,  as  opposed  to  instituting  wholly  new 
development  programs. 

*11.  ***  Select  technology  and  performance  objectives  for  new  developments  con¬ 

servatively  (i.e.,  low  on  the  cost-performance  curve),  except  in  cases  where 
military  necessity  imposes  an  overriding  need  for  risk-taking  to  achieve 
extremes  of  performance.  Allow  for  uncertainty  in  establishing  the  corre¬ 
sponding  system  requirements. 

*12.  ***  Iterate  requirements  and  acquisition  decisions  if  performance,  charac¬ 

teristics,  cost,  quantity,  or  schedule  departs  significantly  from  initial  plans 
during  development.  Establish  criteria  to  trigger  such  iteration. 

13.  (From  ref.  2)  For  mqjor  systems,  start  new-system  acquisition  programs 
with  agency-head  statements  of  needs  and  goals  that  have  been  reconciled 
with  overall  agency  capabilities  and  resources.  State  program  needs  and 
goals  independent  of  any  system  product.  Assign  responsibility  for  respond¬ 
ing  to  statements  of  needs  and  goals  to  a  single  agency  component,  when  the 
mission  need  is  clearly  the  responsibility  of  that  one  component,  or  to  two 
or  more  agency  components  when  competition  is  formally  recognized,  with 
each  offering  alternative  system  solutions  when  the  mission  responsibilities 
overlap. 

14.  (From  ref.  2)  Support  the  general  fields  of  knowledge  that  are  related  to  an 
agency’s  assigned  responsibilities  by  funding  private-sector  sources  and  gov¬ 
ernment  in-house  technical  centers  to  do  basic  and  applied  research,  proof- 
of-concept  work,  and  exploratory  subsystem  development.  Restrict  subsys¬ 
tem  development  to  less  than  fully  designed  hardware  until  identified  as 
part  of  a  system  candidate  to  meet  a  specific  operational  need. 


*Most  directly  applicable  to  project  manager/engineer  day-to-day  undertakings 
***Highest  priority 
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15.  (From  ret  2)  For  mqjor  systems,  create  alternative  system  candidates  by  (a) 
soliciting  industry  proposals  for  new  systems,  with  each  contractor  free  to 
propose  system  technical  approach,  subsystems,  and  main  design  features  on 
the  basis  of  a  statement  of  government  mission,  time,  cost,  and  operational 
requirements;  and  (b)  sponsoring  the  most  promising  system  candidates  as 
selected  by  agency  component  heads,  using  a  team  of  experts  from  inside 
and  outside  the  agency  component  development  organization  as  a  review 
panel. 


DESIGN  TO  COST 


*16.  ***Choose  easily  defined  design-to-cost  (DTC)  targets  such  as  unit  produc¬ 
tion  or  flyaway  costs  (rather  than,  for  example,  the  presently  still  ill-defined 
life-cycle  costs;  but  see  following  subparagraph  to  this  recommendation). 
Establish  such  targets  early,  permitting  successive  revisions  during  develop¬ 
ment,  contractual  commitment  to  a  emit  cost  for  low-rate  initial  production 
at  the  start  of  such  production,  and  another  contractual  commitment  for 
unit  cost  at  the  start  of  full-scale  production  for  systems  to  be  procured  in 
quantity.  Flexibility  to  revise  cost  targets  should  decrease,  and  these  should 
be  based  increasingly  on  firm  experience  as  the  development-to-production 
cycle  progresses. 

If  the  equipment  is  to  be  maintained  by  the  supplier  under  long-term  war¬ 
ranty,  the  design-to-cost  target  can  be  established  as  the  sum  of  the  production  cost 
and  total  warranty  cost;  this  sum  may  be  considered  a  surrogate  for  life-cycle  cost. 
But  if  military  maintenance  is  contemplated,  establishing  life-cycle  cost  as  a  design- 
to-cost  target  is  not  now  appropriate  because  of  the  inadequacy  of  current  knowledge 
of  the  cost  to  the  government  of  militaiy  maintenance,  and  of  the  dependence  of  these 
costs  on  equipment  parameters. 

17.  ***  Establish  explicit  limits  of  deviation  from  “desired”  performance/charac- 
teristics/cost/schedule/quantity  requirements,  and  authorize  program  man¬ 
agers  to  trade  off  freely  among  these  separate  requirement  parameters 
within  the  established  limits.  Establish  “desired”  parameters  and  permis¬ 
sible  deviations  such  that  tradeoffs  are  in  fact  possible  and  not  subject  to 
hidden  constraints  due  to  technical  feasibility,  absolute  force  requirements, 
or  available  budgets. 

18.  ***  To  the  extent  feasible,  maintain  design  and  price  competition  through¬ 
out  the  acquisition  process,  especially  for  components  and  subsystems. 

19.  ***  In  the  contractor-selection  process,  ensure  that  performance  and  cost 
are  considered  together  rather  the  evaluated  separately. 


•Most  directly  applicable  to  project  manager/engineer  day-to-day  undertakings 
** ’Highest  priority 
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420. 


**  This  study  identified  only  one  design-to-cost  acquisition,  namely,  the 
Navy  electronic  warfare  suite,  that  uses  the  approach  of  specifying  equip¬ 
ment  needs  and  requirements  functionally,  leaving  it  to  the  competing 
contractors  to  propose  optimal  development  and  production  strategies  to 
maximize  payoff  to  both  the  government  and  the  contractors,  and  including 
maintenance  strategies  among  the  variables.  More  experience  with  this 
approach  should  be  acquired. 

21.  Increase  the  number  of  design-to-cost  acquisitions  of  electronic  subsystems 
designated  as  “experimental”  for  observation  and  extraction  of  “lessons 
learned.”  In  further  experimental  design-to-cost  acquisitions,  seek  wider 
variation  of  the  management  variables  relevant  to  design  to  cost  (for  exam¬ 
ple,  tradeoffs  among  requirements,  program  manager’s  freedom  to  trade  off, 
types  of  contracts).  The  services  should  publish  “lessons  learned"  periodi¬ 
cally  to  maximize  the  pool  of  explicitly  analyzed  experience  available  to  all. 

22.  **  Review  the  contracting  procedures  associated  with  design-to-cost  con¬ 
tracts,  modify  those  that  inhibit  requisite  design-to-cost  flexibility,  and 
incorporate  the  modifications  in  the  FAR,  if  necessary. 

DESIGN  FOR  IMPROVED  RELIABILITY 


*23.  ***  Limit  the  complexity  of  new  subsystem  or  equipment  designs  (as  mea¬ 

sured  by  criteria  such  as  unit  production  cost  or  parts  count)  to  a  level  con¬ 
sistent  with  the  reliability  required  by  a  mission  analysis.  Require  evidence 
of  compliance  as  a  preliminary  DSARC  review  for  electronic  subsystems  of 
mqjor  systems,  and  as  a  preliminary  to  sub-DSARC  review  for  independently 
developed  electronic  subsystems. 

*24  ***  Require  contractually  the  in-plant  use  of  a  formal  management  method¬ 

ology,  such  as  methods  using  Duane-curve  monitoring,  to  ensure  reliability 
growth  in  electronic  equipment  and  systems. 

*25.  ***  Use  long-term  contractor  maintenance  warranties  to  motivate  the  con¬ 

tractor  to  design  for  minimum  life-cycle  cost.  (See  later  recommendations  on 
warranties  for  further  details.) 

*26.  ***  Specify  the  reliability  of  electronic  equipments  or  systems  to  be  consis¬ 

tent  with  predictions  based  on  their  anticipated  complexity  (or  unit  produc¬ 
tion  cost,  as  a  surrogate  for  complexity). 

27.  ***  Undertake  redesign  of  selected  equipments  with  the  specific  objective  of 

improving  reliability  while  holding  performance  constant. 


•Most  directly  applicable  to  project  manager/engineer  day-to-day  undertakings 
**High  priority 
“‘Highest  priority 
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DESIGN  TO  FACILITATE  COMPETITION 


*28.  ***  Lay  the  groundwork  for  future  design  and  price  competition  through 

production  and  for  ready  replacement  of  old  designs  by  new-generation 
equipment  by  ensuring  the  interchangeability  of  similar  equipments 
intended  for  similar  applications.  Accomplish  this  by  including  mechanical, 
electrical,  and  environmental  interface  standards  for  each  unit  as  part  of 
military  electronic  equipment  specifications. 

Require  design  interchangeability  when  production  competition  or  design 
upgrading  is  foreseen  as  desirable  or  likely.  Equipment  classes  that  are  judged  ripe 
for  initial  application  of  interface  standardization  are  airborne  communication,  navi¬ 
gation,  identification,  and  weather  radar;  vehicular  communication;  and  modular 
electronics  packages  for  tactical  missiles. 

*29.  ***  Modify  approval  processes  for  engineering  change  proposals  to 

expedite  incorporation  by  suppliers  of  internal  design  improvements  to 
enhance  reliability  and  performance  or  inclusion  of  new  technology  to  meet 
competition  during  the  procurement  cycle  and  even  after  deployment,  if  the 
suppliers  are  called  upon  to  maintain  their  equipment.  But  keep  rigid  con¬ 
trol  over  interface  configurations  to  ensure  interchangeability. 

30  ***  Obtain  multiple  developments  of  equipments  conforming  to  interface 

specifications.  Where  the  potential  market  for  the  equipment  is  large 
enough,  encourage  industry-financed  development;  otherwise,  procure  multi¬ 
ple  developments  under  government  contracts. 

31.  ***  Facilitate  government  testing  and  qualification  of  designs  offered  in 

compliance  with  the  specifications,  whether  or  not  the  designs  were 
developed  under  government  contract.  Plan,  prepare,  and  provide  for  retest¬ 
ing  and  requalification  of  modified  designs  submitted  in  production  competi¬ 
tions  subsequent  to  the  initial  competition. 

*32.  ***  To  overcome  the  potential  problem  of  spare-parts  stocking  and  field 

repair  of  multiple  equipment  configurations,  make  use  of  depot  repair  or 
supplier  maintenance  under  warranty.  In  the  field,  replace  rather  than 
repair  failed  replaceable  units  of  equipment.  Include  warranty  requirements 
when  initiating  development. 

*33.  **  To  achieve  multiple-source  availability,  rely  on  performance  specifica¬ 

tions  plus  environmental  and  interface  requirements  (i.e.,  “form,  fit,  func¬ 
tion”  specifications)  to  define  equipment,  rather  than  imposing  detailed 
specifications  on  parts,  processes,  materials,  and  internal  configuration. 


*Most  directly  applicable  to  project  manager/engineer  day-to-day  undertakings 
“High  priority 
•“Highest  priority 
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34.  To  broaden  the  markets  for  competitive  suppliers,  encourage  the  evolution 
of  multiservice  interface  standards. 

35.  (From  ref.  2)  Maintain  competition  between  contractors  exploring  alterna¬ 
tive  systems  by  (a)  limiting  commitments  to  each  contractor  to  annual 
fixed-level  awards,  subject  to  annual  review  of  their  technical  progress; 

(b)  assigning  agency  representatives  to  advise  competing  contractors  in 
developing  requirements  for  each  candidate  system  as  tests  and  tradeoffs  are 
made;  and  (c)  concentrating  the  activities  of  agency  development  organiza¬ 
tions,  government  laboratories,  and  technical  management  staffs  on  moni¬ 
toring  and  evaluating  contractor  efforts,  and  on  participating  in  those  tests 
critical  to  determining  whether  the  candidate  system(s)  should  be  continued. 

36.  (From  ref.  2)  Limit  premature  major-system  commitments  and  retain  the 
benefit  of  competition.  This  should  be  accomplished  by  conducting  competi¬ 
tive  demonstrations  of  candidate  systems  after  a  determination  by  agency 
heads  to  do  so  and  on  these  stem:  (a)  choosing  contractors  according  to  their 
relative  technical  progress,  the  remaining  uncertainties,  and  the  economic 
constraints;  (b)  providing  selected  contractors  with  the  operational  test  con¬ 
ditions,  mission  performance  criteria,  and  lifetime  ownership  cost  factors  to 
be  used  in  the  formal  system  evaluation  and  selection;  (c)  proceeding  with 
final  development  and  initial  production,  and  with  commitments  to  a  firm 
date  for  operational  use  (after  agency  needs  and  goals  are  reaffirmed,  com¬ 
petitive  demonstration  results  prove  that  the  chosen  technical  approach  is 
sound,  and  the  definition  of  a  procurement  program  is  practicable);  and 

(d)  strengthening  each  agency’s  cost  estimating  capability. 

37.  (From  ref.  2)  Should  an  agency  component  (responsible  developer)  determine 
it  should  concentrate  development  resources  on  a  single  system  rather  than 
fund  competitive  system  candidates,  require  agency  head  approval  before 
proceeding.  When  such  single  system  development  is  to  be  pursued, 

(a)  establish  a  strong  centralized  program  office  within  the  agency  comp- 
nent  to  take  direct  technical  and  management  control  of  the  program; 

(b)  integrate  selected  technical  and  management  contributions  from 
in-house  groups  and  contractors;  (c)  select  contractors  with  proven  manage¬ 
ment,  financial,  and  teclinical  capabilities,  as  related  to  the  problem,  and 
use  cost-reimbursement  contracts  for  high-technical-risk  portions  of  the  pro¬ 
gram;  and  (d)  estimate  program  cost  within  a  probable  range  until  the  mqjor 
system  reaches  the  final  development  phase. 

38.  (From  ref.  2)  Withhold  agency  head  approval  and  congressional  commit¬ 
ments  for  full  production  and  use  of  new  mqjor  systems  until  the  need  has 
been  reconfirmed  and  the  system  performance  has  been  tested  and  evaluated 
in  an  environment  that  closely  approximates  expected  operational  condi¬ 
tions.  Regarding  T&E,  (a)  continue  efforts  to  strengthen  test  and  evaluation 
capabilities  in  the  services,  (b)  establish  an  agency-wide  definition  of  the 


“High  priority 
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scope  of  operational  test  and  evaluation,  and  (c)  establish  in  each  agency 
component  an  operational  T&E  activity  separate  from  its  developer  and  user 
organizations. 

39.  (From  ref.  2)  Use  contracting  as  an  important  tool  of  system  acquisition,  not 
as  a  substitute  for  management  of  acquisition  programs.  In  so  doing,  (a)  set 
policy  guidelines  within  which  experienced  personnel  may  exercise  judgment 
in  selectively  applying  detailed  contracting  regulations,  (b)  develop  simpli¬ 
fied  contractual  arrangements  and  clauses  for  use  in  awarding  final  develop¬ 
ment  and  production  contracts  for  demonstrated  systems  tested  under 
competitive  conditions,  and  (c)  allow  contracting  officials  to  use  priced  pro¬ 
duction  options  if  critical  test  milestones  have  reduced  risk  to  the  point  that 
the  remaining  development  work  is  relatively  straightforward. 

40.  (From  ref.  2)  Within  each  agency  and  agency  component,  unify  policy¬ 
making  and  monitoring  responsibilities  for  system  acquisitions,  including 
minimization  of  management  echelons,  staff  reviews,  coordinating  points, 
procedures,  reporting,  and  paperwork  within  both  industry  and  government. 

PRODUCTION 

41.  “*  Where  the  quantity  to  be  bought  is  large  enough,  depart  from  the  con¬ 
ventional  approach  of  aggregating  procurements  into  a  single  large  buy 
intended  to  take  advantage  of  “learning  curves."  Instead,  fragment  the 
procurements  into  sequential  buys,  inviting  design  and  price  competition  on 
each  buy  by  the  several  suppliers  of  qualified  interchangeable  equipments. 

42.  “  The  government  must  assure  prospective  suppliers  that  there  will  be 
future  design  and  price  competitions.  One  method  of  doing  so  is  to  analyze 
and  publish  future  needs  and  a  schedule  of  planned  competitive  buys. 

43.  ***  The  government  must  provide  assurance  that  new  or  improved 
designs  will  be  given  full  consideration  in  future  competitions  if  they  meet 
the  form,  fit,  and  function  requirements  that  ensure  interchangeability  with 
prior  designs.  This  implies  the  need  for  inclusion  of  interface  requirements 
in  government  specifications. 

44.  “*  The  government  must  offer  to  perform  and  must  be  prepared  to  perform 
laboratory  tests  and  evaluations  of  the  actual  hardware  prototypes  offered 
by  bidders  or  prospective  competitors  in  order  to  qualify  the  designs  for  cur¬ 
rent  and  future  competition. 

45.  When  it  is  desirable  and  necessary  to  sustain  competition,  award  fractions 
of  each  buy  to  two  or  three  competitors  in  proportion  to  the  merit  of  their 
respective  designs  and  prices,  rather  than  making  the  award  on  a  winner- 
take-all  basis. 
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REPROCUREMENT 


46.  **  In  selected  development  contracts  where  subsequent  competitive 
reprocurement  is  anticipated,  the  government  should  provide  a  payment  to 
the  developer  for  each  accepted  unit  produced  under  government  contract 
from  the  developer’s  design  by  a  supplier  other  than  the  developer.  This  pay¬ 
ment  should  constitute  a  deferred  part  of  the  compensation  for  the  reprocu¬ 
rement  data  package.  This  contracting  procedure  should  be  used  by  the 
government  on  a  trial  basis. 

MAINTENANCE 

47.  ***  As  recommended  earlier,  institute  a  cost  accounting  system  that  will 
afford  visibility  of  the  maintenance  process  and  make  possible  realistic  cost 
comparisons  between  military  and  industrial  maintenance. 

48.  **  Provide  separate  accounts  for  functions  other  than  maintenance,  such  as 
the  use  of  continental  US  maintenance  billets  to  facilitate  the  rotation  of 
military  personnel  not  involved  in  maintenance,  or  for  personnel  in  training 
from  overseas  and  shipboard  billets. 

*49.  ***  Establish  alternative  sources  of  maintenance,  including  the  maximum 

feasible  amount  of  contractor  maintenance,  to  foster  competition  and  resul¬ 
tant  efficiency  in  the  maintenance  process  and  to  ensure  the  proper  utiliza¬ 
tion  of  scarce  military  personnel  in  the  present  zero-draft,  environment. 

*50.  **  Intensify  efforts  to  reduce  field  maintenance  by  shifting  complex  tasks 

from  the  organizational  and  intermediate  levels  to  the  depots,  taking  due 
account  of  increased  turnaround  time  and  transportation  problems. 

MAINTENANCE  TRAINING 

51.  ***  Develop  fully  proceduralized  job  performance  aids  for  use  in  routine 
maintenance  of  new  weapon  systems  and  for  selected  tasks  in  high- 
maintenance  portions  of  existing  systems. 

52.  ***  Selectively,  on  a  trial  basis,  reorient  the  training  sequence  for  electronic 
technicians  so  as  to  provide  first  the  specific  training  they  require  to  per¬ 
form  maintenance  tasks  by  using  procedurialized  aids  during  their  initial 
enlistments. 


‘Most  directly  applicable  to  project  manager/engineer  day-to-day  undertakings 
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53.  Increase  research  on  job  performance  aids  and  on  job-oriented  training  to 
enable  the  utilization  of  personnel  of  lower  ability  levels  and  to  enhance 
learning  on  the  job.  Apply  the  results  in  selected  training  programs. 

WARRANTIES 

*54.  ***  Extend  the  application  of  long-term  contractor  maintenance  warranties 

to  military  electronics  procurements. 

*55.  ***  Make  known  the  intention  to  contract  for  maintenance  warranties  on 

production  equipment  at  the  time  development  is  initiated,  so  that  the  con¬ 
tractor  will  design  to  minimize  total  costs  of  production  and  warranty  main¬ 
tenance. 

5G.  ***  Establish  a  warranty  review  group  within  OSD  to  monitor  results  of 

trial  applications,  to  determine  desirable  warranty  contractual  formats,  and 
to  refine  the  categories  of  equipments  to  which  warranties  are  most  applica¬ 
ble  and  for  which  warranties  are  most  effective. 

*57.  ***  Initially,  apply  long-term  contractor  maintenance  warranties  to  equip¬ 

ments  whose  failed  units  can  be  replaced  in  the  field  and  conveniently 
returned  to  the  contractor’s  plant  or  base  for  repair,  or  to  which  the  con¬ 
tractor  can  have  ready  access  for  field  repair,  such  as  airborne  communica¬ 
tions,  navigation,  and  identification  equipment;  modular  radars;  vehicular 
communication  sets;  complex  manpack  equipment  such  LORAN  C/D;  for¬ 
ward-looking  infrared  (FLIR)  systems;  and  domestic,  communication,  data 
processing,  and  radar  installations. 

DESIGN  EVOLUTION  AND  CONFIGURATION  MANAGEMENT 

58.  ***  The  recently  promulgated  DoD  regulation  on  configuration  management 

(NAVMATINST  4130.1A,  for  the  Navy)  should  be  adopted  with  the  follow¬ 
ing  modifications:  (a)  specifically  permit  consideration  of  changes  that  are  of 
benefit  to  the  contractor  and  not  detrimental  to  the  government;  (b)  estab¬ 
lish  two  product  baselines,  the  first  (tentative)  one  at  the  end  of  full-scale 
development,  and  the  second  (final)  one  when  the  design  has  been  ade¬ 
quately  stabilized;  and  (c)  permit  internal  equipment  changes  that  do  not 
affect  form,  fit  (compatibility  and  interfaces),  function,  price,  or  delivery  to 
be  classified  Class  II  (defined  in  the  regulation)  in  order  to  facilitate  the 
change  approval  process  until  the  “final”  product  baseline  is  invoked  by  the 
government. 


*Most  directly  applicable  to  project  manager/engineer  day-to-day  undertakings 
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59.  ***  The  government  should  defer  invocation  of  the  final  product  baseline,  as 

applicable  to  electronic  equipment,  until  field  reliability  objectives  have 
been  achieved,  or,  in  the  case  of  equipment  under  contract  maintenance 
warranty,  until  the  warranty  period  is  about  to  end  and  the  government  is 
about  to  take  over  maintenance  from  the  warrantor. 

*60,  ***  The  govemn  ent  should  defer  full  spares  stocking  until  after  the  final 

product  baseline  is  invoked. 

PROJECT  MANAGEMENT 

61.  ***  Use  the  multiprogram  project  office  (“basket”  SPO)  structure  for  all 
independent  electronic  subsystem  developments  where  a  number  of 
related  or  similar  developments  can  be  grouped  under  one  perpetual  project 
manager  (PM)  to  provide  a  PM  of  higher  rank  and  greater  authority,  better 
project  office  personnel,  more  responsive  support  from  functional  groups, 
and  more  tradeoff  flexibility. 

62.  *“  Provide  multiprogram  project  offices  with  sufficient  flexibility  in  the 
use  of  available  R&D  funds  to  allow  the  necessary  tradeoffs  by  the  PM  in 
the  development,  operational  test  and  evaluation,  and  low-rate  initial  pro¬ 
duction  phases. 

S3.  **  Arrange  for  the  project  manager  or  prospective  pioject  manager  to  par- 

ticipate  in  drafting  the  operational  requirements  before  developing  specifica¬ 
tions  for  subsystems  under  his  jurisdiction. 

64.  **  Make  available  to  system  project  managers  catalogs  of  available  elec¬ 
tronic  equipment  that  show  current  price  and  reliability  figures  as  well  as 
technical  descriptions. 

STANDARDIZATION  AND  SPECIFICATIONS 

65.  ***  DoD  should  establish  an  Electronic  Standards  Panel  having  responsibil¬ 
ity  and  authority  to  act  on  recommendations  66  through  76,  following. 

66.  ***  Promulgate  policy  requiring  that  the  services  include  electrical, 
mechanical,  and  environmental  interface  specifications  in  specifications  for 
electronic  equipment. 

67.  **  Promulgate  policy  requiring  that  the  services  take  steps  toward  assuring 
that  new  electronic  equipments  that  are  likely  to  replace  older  equipments 
in  aircraft,  ground  vehicles,  and  other  platforms  wiU  be  made  electrically, 
mechanically,  and  environmentally  interchangeable  with  the  older  equip¬ 
ments,  of  similar  types,  so  that  the  new  equipments  can  be  substituted  for 
the  old  without  costly  installation  modification. 


*Most  directly  applicable  to  project  manager/engineer  day-to-day  undertakings. 
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68.  ***  Promulgate  policy  requiring  that  equipments,  subsystems,  or  systems  of 
similar  types  be  developed  to  the  same  interface  specifications,  so  that  they 
may  be  interchanged. 

69.  “  Promulgate  specific  interface  standards  for  classes  of  equipment  used  by 
more  than  one  service. 

70.  “  Establish  and  promulgate  standards  for  the  thermal,  atmospheric, 
vibration,  shock,  mounting,  shielding,  and  power-source  environments  to  be 
provided  by  aircraft,  ships,  and  vehicles  in  which  electronic  equipment  is  to 
be  installed.  This  should  include  standards  for  benign-environment  enclo¬ 
sures  whenever  they  are  feasible  and  cost-effective. 

71.  **  With  the  concurrence  of  and  to  the  extent  authorized  by  the  Military 
Communications  Electronics  Board,  establish  and  promulgate  standards  for 
the  signals  to  be  transmitted  or  interchanged  in  cooperative  systems,  such 
as  communications,  navigation,  and  identification  systems. 

72.  “  Review  service  forecasts  of  electronic  equipment  needs  in  order  to 
determine  those  types  and  classes  to  which  uniform  standards  should  be 
applied,  and  act  to  ensure  that  the  standards  are  applied. 

73.  *“  Establish  and  promulgate  DoD  standards  for  the  multiplexing  and  inter¬ 
change  of  digital  data  among  electronic  equipments  within  ships  and  air¬ 
craft. 

74.  “  Promulgate  policy  designed  to  ensure  maximum  compatibility  of  military 
standards  with  commercial  practices. 

75.  “  Review  existing  standards  and  specifications  for  parts,  materials,  fin¬ 
ishes,  processes,  and  other  aspects  of  the  internal  design  of  military  elec¬ 
tronics  to  determine  which  should  be  (a)  strictly  enforced,  (b)  subject  to  the 
substitution  of  the  contractor-validated  alternative,  (c)  regarded  as  advisory 
only,  or  (d)  revoked.  The  several  general  design  specifications  used  in  most 
electronics  procurements  (e.g.,  MIL-E-16400,  MIL-E-5400,  MIL-I-983) 
should  receive  particular  early  attention. 

76.  “  Issue  up-to-date  guidance  on  military  utilization  of  standard  commercial 
LSI  and  MSI  items,  with  particular  attention  to  the  need  for  multiple 
sources  and  avoidance  of  military-unique  designs. 

77.  “  DoD  Directive  4120.3  (DoD  Standardization  Program)  can  be  the  vehicle 
for  the  establishment  of  an  effective  electronic  standards  organization.  In 
order  to  accomplish  this,  the  defense  Material  Specifications  and  Standards 
Board  should,  under  paragraph  VII  B2  of  the  directive,  recommend  the 
establishment  of  an  Electronic  Standards  Panel  (ESP)  with  the  authority 
and  responsibility  to  promulgate  multiservice  electronic  standards  and  pro¬ 
mote  the  cause  of  standardization  of  electronic  equipments,  subsystems,  and 
systems,  both  single-service  and  multiservice.  The  ESP  should  be  given  the 
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further  authority  to  establish  continuing  (as  opposed  to  ad  hoc)  committees, 
to  which  may  be  delegated  segments  of  the  authority  and  responsibility  of 
the  ESP.  Once  established,  the  ESP  should  organize  to  undertake  formula¬ 
tion  and  promulgation  of  the  policies  recommended  above. 


SOFTWARE 


To  reduce  costs  of  software  in  processors  employing  conventional  general- 
purpose  machines: 

*78.  ***  Complete  the  design  of  the  system  and  the  basic  program  structure  in 

substantial  detail  before  making  major  commitments  to  hardware  or  coding. 

*79.  **  Limit  the  aggregation  of  problems  to  be  solved  on  a  central  machine;  as 

an  alternative,  decentralize  processing  by  providing  peripheral  special- 
purpose  devices  (either  analog  or  digital)  or  separate  peripheral  general- 
purpose  machines  to  perform  specific  separable  functions. 

*80.  **  Select  a  processor  of  adequate  size  to  permit  underutilizing  the  com¬ 

puter;  write  highly  modular  programs;  emphasize  structure  and  overall  effi¬ 
ciency  rather  than  hardware  efficiency  alone. 

*81.  **  Use  rigorous  discipline  in  software  development,  such  as  the  top-down 

structured-programming  approach. 

*82.  **  Use  a  standard,  well-established  programming  language  with  which  pro¬ 

grammers  are  thoroughly  familiar.  Use  the  highest-level  language  appropri¬ 
ate  to  the  task  at  hand,  but  avoid  the  unnecessary  development  of  a  unique 
language. 

*83.  **  Defer  coding  until  the  computer  design  is  substantially  complete  and 

firm,  except  for  that  necessary  to  verify  hardware-software  design  compati¬ 
bility. 


DIGITAL  SYSTEMS  ARCHITECTURE 

*84.  ***  Systoms-function-oriented  processing  hardware  structures  should  be 

considered  as  alternatives  to  the  conventional  centralized  programmable 
uniprocessor  for  use  in  military  tactical  systems. 

*85.  ***  The  military  processing  problem  should  be  clearly  stated,  the  system 

design  should  be  spelled  out  in  detail  and  alternate  processor  architectures 
and  designs  should  be  compared  before  a  hardware  approach  is  selected. 


*Most  directly  applicable  to  project  manager/engineer  day-to-day  undertakings 
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*86.  ***  A  processor  design  for  each  system  should  be  selected  and  developed 

that  will  minimize  the  combined  costs  of  hardware  and  software;  the  alloca¬ 
tion  of  functions  between  hardware,  software,  and  human  operators  should 
be  conscientiously  worked  out  prior  to  decision. 

*87.  **  Standard  LSI  processing  elements  available  from  more  than  one  source 

should  be  used  to  the  maximum  extent  possible;  development  of  uniquely 
military  LSI  elements  should  be  minimized. 

88.  **  Military  laboratories  should  be  encouraged  to  investigate  and  develop 

processor  architectures,  including  federated  architectures,  that  fit  military 
problems  and  are  cost-effective.  Conversely,  their  extensive  efforts  in  the 
programming  of  conventional  uniprocessors  should  be  reduced  to  bring  over¬ 
all  programs  into  better  balance. 

*89.  **  Commercially  successful  processors  for  which  software  already  exists 

should  be  considered  for  DoD  applications  wherever  appropriate. 

90.  **  Formats  and  speeds  for  data  interchange  among  sensors,  actuators,  proc¬ 

essors,  controls,  and  displays  should  be  standardized  across  service  lines  and 
for  as  wide  a  variety  of  applications  as  practicable. 

DATA  COSTS 

*91.  Accept  contractor’s  data  format  unless  there  is  a  demonstrable  advantage  in 
specifying  a  government  format. 

*92.  *•  Defer  the  ordering  and  delivery  of  contractor  data  until  the  need  is  firmly 

established. 

*93.  “Delay  procurement  of  spares  provisioning,  technical  manuals,  and  mainte¬ 
nance  handbooks  until  the  point  of  design  stabilization  is  identified  and 
reached. 

*94.  **  Scrub  data  requirements  mercilessly  through  the  efforts  of  Data  Require¬ 

ments  Review  Boards  that  include  representation  of  the  project  manager, 
the  user,  and  industry. 

*95.  Where  the  equipment  future  is  uncertain,  buy  options  on  reprocurement 
data  instead  of  the  data  items  themselves. 

96.  (From  ref.  2)  Establish  standards  and  criteria  for  estimating  costs  and  bene¬ 
fits  of  product  data  requirements.  The  need  for  product  data  should  be 
determined  on  the  basis  of  cost-benefit  analyses.  Selective  after-the-fact 
reviews  should  be  used  as  the  basis  for  eliminating  unnecessary  require¬ 
ments. 
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97.  (From  ref.  2)  Establish  government-wide  criteria  for  management  systems 
which  are  prescribed  for  use  by  contractors,  including  standards  for  deter¬ 
mining  mission-essential  management  data  requirements. 

Following  recommendations  98-101  are  rewordings  of  previous  recommenda¬ 
tions  in  the  context  of  data  costs. 

98.  Use  competing  suppliers  of  interchangeable  equipment  to  reduce  the  need 
for  reprocurement  data. 

*99.  Use  contractor  warranties  and  maintenance  to  reduce  the  need  for  technical 
and  maintenance  manuals  and  provisioning  data. 

*100.  Rely  on  competitive  prototyping  and  test  as  a  substitute  for  voluminous 

in-process  validation  data  (and  as  a  substitute  for  myriad  detailed  specifica¬ 
tions). 

*101.  As  an  alternative  to  formal  and  highly  detailed  reprocurement  drawings  and 
specifications,  require  less-formal  drawings  and  encourage  more  informal 
information  transfer.  For  reprocurement  data,  pay  a  fixed  amount  for  the 
drawings  plus  a  fixed  amount  for  each  equipment  successfully  delivered  by 
the  second  source. 

GENERAL  PROCUREMENT  CONSIDERATIONS 

102.  (From  ref.  5)  To  provide  an  open  environment  in  which  recommended 
changes  can  take  place,  the  hierarchy  of  DoD  program  management  struc¬ 
tures  should  be  realigned  and  simplified. 

103.  (From  ref.  2)  Require  the  use  of  formal  advertising  when  the  number  of 
sources,  the  existence  of  adequate  specifications,  and  other  conditions  jus¬ 
tify  such  use.  Authorize  the  use  of  competitive  negotiation  methods  as  an 
acceptable  and  efficient  alternative  to  formal  advertising.  Require  that  the 
procurement  file  disclose  the  reasons  for  using  competitive  methods  other 
tlian  formal  advertising,  in  procurements  over  $10,000  or  such  other  figure 
as  may  be  established  for  small-purchase  procurers.  Repeal  statutory  provi¬ 
sions  inconsistent  with  these  proposed  requirements. 

104.  (From  ref.  5)  The  important  role  of  cost-plus-fixed-fee  contracts  should  con¬ 
tinue  for  development  and  prototype  contracts  where  effective  fixed-price 
competition  cannot  be  achieved  without  the  addition  of  large  contingency 
factors. 

105.  (From  ref.  2)  The  procurement  of  professional  services  should  be  accom¬ 
plished,  so  far  as  practicable,  by  using  competitive  proposal  and  negotiation 
procedures  which  take  into  account  the  competence  of  the  proposers,  the 
proposed  concept  of  the  end  product,  and  the  estimated  cost  of  the  project, 
including  fee.  The  primary  factors  in  the  selection  process  should  be  the 
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professional  competence  of  those  who  will  do  the  work,  and  the  relative 
merits  of  proposals  for  the  end  product,  including  cost,  sought  by  the  gov¬ 
ernment.  The  fee  to  be  charged  should  not  be  the  dominant  factor  in  con¬ 
tracting  for  professional  services. 

106  (From  ref.  2)  Provide  through  legislation  that  it  is  national  policy  to  rely  on 
private  enterprise  for  needed  goods  and  services,  to  the  maximum  extent 
feasible  and  within  the  framework  of  procurement  at  reasonable  prices. 


B-16 


